-- Logs begin at Tue 2022-07-26 06:44:28 UTC, end at Tue 2022-07-26 08:14:50 UTC. -- Jul 26 07:10:39.197620 dv-cinder-850836-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 26 07:10:39.207944 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 26 07:10:39 2022] *** Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: nodename: dv-cinder-850836-1-iscsi Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: machine: x86_64 Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: clock source: unix Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: pcre jit disabled Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: detected number of CPU cores: 4 Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: current working directory: / Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: detected binary path: /usr/bin/uwsgi-core Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: your processes number limit is 47680 Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: your memory page size is 4096 bytes Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: detected max file descriptor number: 2048 Jul 26 07:10:39.209960 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: lock engine: pthread robust mutexes Jul 26 07:10:39.210698 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: thunder lock: enabled Jul 26 07:10:39.210698 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 26 07:10:39.210698 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 26 07:10:39.249228 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: Python main interpreter initialized at 0x55e2bd1889c0 Jul 26 07:10:39.249228 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: python threads support enabled Jul 26 07:10:39.249228 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: your server socket listen backlog is limited to 100 connections Jul 26 07:10:39.249228 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: your mercy for graceful operations on workers is 90 seconds Jul 26 07:10:39.249601 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: mapped 403077 bytes (393 KB) for 2 cores Jul 26 07:10:39.249670 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: *** Operational MODE: preforking *** Jul 26 07:10:39.249958 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: *** uWSGI is running in multiple interpreter mode *** Jul 26 07:10:39.249958 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: spawned uWSGI master process (pid: 80159) Jul 26 07:10:39.250177 dv-cinder-850836-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 26 07:10:39.251218 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: spawned uWSGI worker 1 (pid: 80169, cores: 1) Jul 26 07:10:39.251218 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: spawned uWSGI worker 2 (pid: 80170, cores: 1) Jul 26 07:10:39.251218 dv-cinder-850836-1-iscsi devstack@keystone.service[80159]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 26 07:10:44.700863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 07:10:44.700863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Jul 26 07:10:44.701654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 07:10:44.701654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jul 26 07:10:44.702087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 07:10:44.702087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: from cryptography.utils import int_from_bytes Jul 26 07:10:44.702186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 26 07:10:44.702186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: from cryptography.utils import int_from_bytes Jul 26 07:10:44.750953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 07:10:44.751130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 26 07:10:44.751221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 07:10:44.751297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 26 07:10:44.751357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] command line args: [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 07:10:44.751494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] command line args: [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 26 07:10:44.751580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 07:10:44.751764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 26 07:10:44.751897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ================================================================================ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 07:10:44.751972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ================================================================================ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 26 07:10:44.752030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.752166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.752244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_dir = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.752442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_dir = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.752565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.752638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] config_source = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.752707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.752777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] debug = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.752841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] config_source = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.752992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] debug = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.753148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.753408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.753507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.753684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.753843 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.753843 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.753974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.754100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.754176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.754385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.754509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.754578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_config_append = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.754646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.754710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.754767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_config_append = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.754904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_dir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.754980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.755038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.755103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_dir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.755281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.755392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.755501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.755576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.755747 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.755863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.756048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.756170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.756170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.756322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.756395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.756479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.756563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.756640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.756792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.756873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.757002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.757126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.757327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.757456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.757456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.757652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.757776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.757862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.757942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.758066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.758218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.758388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.758487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.758567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.758715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.758983 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.759142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] publish_errors = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.759242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.759395 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] publish_errors = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.759514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.759601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.759675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.759793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.759906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.760025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.760135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.760264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.760387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.760499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] standard_threads = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.760615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.760733 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.760899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] standard_threads = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.761012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.761122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.761238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.761355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.761465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_journal = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.761585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.761696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_json = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.761833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_journal = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.761957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_stderr = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.762069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_json = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.762187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] use_syslog = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.762298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_stderr = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.762408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.762524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] use_syslog = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.762524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 26 07:10:44.762730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.762856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.762968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.763085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.763201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.763318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.763429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.763551 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.763661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.763777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.763889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.763998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.764108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.764226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.764342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.764452 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.764561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.764671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.764786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.764908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.765032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.765142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.765258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.765368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.765477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.765598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.765715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.765859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.765973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.766089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.766206 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.766394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.766514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.766626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.766735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.766845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.766955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.767074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.767185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.767294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.767410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.767521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.767630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.767756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.767869 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.767979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.768089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.768198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.768308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.768426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.768538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.768653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.768769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.768885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.769002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.769123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.769234 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.769350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.769461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.769576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.769699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.769832 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.769959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.770134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.770277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.770390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.770510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.770636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.770747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.770857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.770966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.771075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.771197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.771310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.771419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.771528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.771638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.771752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.771940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.772053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.772163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.772274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.772383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.772493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.772611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.772733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.772844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.772953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.773062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.773177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.773296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.773414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.773531 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.773641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.773811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.773925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.774048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.774160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.774255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.774348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.774447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.774542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.774651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.774746 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.774854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.774955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.775059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.775190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.775309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.775433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.775536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.775645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.775762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.775873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.775992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.776104 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.776215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.776326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.776435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.776546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.776656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.776776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.776888 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.777009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.777121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.777240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.777362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.777474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.777586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.777712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.777880 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.777995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.778106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.778217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.778338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.778450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.778562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.778673 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.778784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.778896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.779055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.779167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.779285 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.779397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.779508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.779631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.779743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.779854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.779964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.780090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.780246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.780396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.780518 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.780630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.780741 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.780853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.780963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.781082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.781195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.781305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.781415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.781526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.781643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.781818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.781935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.782053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.782164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.782281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.782393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.782512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.782624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.782736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.782846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.782962 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.783073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.783257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.783380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.783492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.783602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.783837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.783959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.784081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.784203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.784327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.784440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.784551 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.784661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.784778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.784891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.785010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.785125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.785237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.785348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.785458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.785568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.785690 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.785836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.785963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.786082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.786194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.786305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.786427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.786540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.786657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.786769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.786880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.787006 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.787186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.787315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.external = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.787428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.external = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.787539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.787650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.787769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.787909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.788038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.788152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.788263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.password = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.788380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.password = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.788492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] auth.token = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.788604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] auth.token = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.788728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.788841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.788952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.789064 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.789176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.789296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.789417 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.789536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.789649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.789803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.789917 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.790029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.790154 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.790269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.790372 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.790483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.790593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.790714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.790835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.790948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.791059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.791171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.791282 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.791394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.791514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.791626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.791744 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.791857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.791975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.792086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.792207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.792320 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.792431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.792548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.792659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.792771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.792890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.793002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.793113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.793225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.793335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.793447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.793567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.793680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.793816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.793938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.794051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.794163 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.794284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.794402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.794514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.794625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.794736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.794847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.794967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.795083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.795196 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.795307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.795420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.795522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.795625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.795722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.795834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.795950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.796064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.796175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.796295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.796410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.796521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.796632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.796743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.796863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.797051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.797172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.797285 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.797402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.797516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.797636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.797769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.797884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.797995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.798106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.798226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.798350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.798464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.798575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.798796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.798907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.799026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.799026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.799236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.799347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.799457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.799569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.799689 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.799802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.799913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.800023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.800134 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.800245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.800367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.800480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.800593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.800705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.800815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.800927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.801046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.801159 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.801270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.801381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.801492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.801602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.801722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.801975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.802088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.802200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.802312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.802424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.802549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.802665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.802778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.802890 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.803001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.803112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.803233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.803352 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.803464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.803576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.803694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.803808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.803929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.804042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.804154 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.804266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.804377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.804489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.804610 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.804723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.804836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.804947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.805063 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.805183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.805308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.805422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.805534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.805646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.805779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.805894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.806018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.806132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.806243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.806354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.806504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.806615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.806735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.806849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.806961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.807073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.807185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.807296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.807416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.807529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.807641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.807752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.807863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.807976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.808096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.808209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.808320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.808431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.808542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.808654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.808774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.808887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.809000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.809112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.809223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.809343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.809456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.809568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.809679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.809804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.809917 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.810166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.810279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.810390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.810501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.810612 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.810731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.810844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.810956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.811068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.811179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.811290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.811413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.811527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.811638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.811748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.811859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.811971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.812091 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.812205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.812317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.812429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.812541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.812652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.812776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.812890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.813001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.813113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.813225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.813337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.813457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.813576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.813688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.813871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.813986 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.814098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.814220 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.814333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.814445 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.814556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.814668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.814779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.814899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.815118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.815270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.815413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.815556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.815699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.815853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.815997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.816139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.816290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.816447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.816599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.816761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.816912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.817062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.817211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.817362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.817513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.817674 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.817852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.818004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.818157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.818321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.818471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.818631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.818775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.818917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.819059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.819201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.819343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.819498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.819647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.819808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.819965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.820121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.820284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.820541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.820702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.820862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.821053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.821171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.821283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.821405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.821519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.821632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.821771 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.821886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.821999 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.822123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.822238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.822350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.822461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.822573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.822685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.822805 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.822918 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.823029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.823140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.823251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.823371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.823484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.823595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.823707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.823825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.823938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.824058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.824171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.824283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.824395 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.824506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.824617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.824738 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.824851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.824963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.825078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.825192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.825304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.825424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.825538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.825649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.825783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.825898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.826010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.826133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.826247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.826358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.826470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.826580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.826692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.826813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.826926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.827038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.827150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.827150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.827359 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.827479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.827592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.827703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.827814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.827925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.828045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.828157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.828269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.828380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.828491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.828602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.828722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.828835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.828947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.829058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.829168 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.829287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.829407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.829520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.829631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.829760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.829874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.829987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.830111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.830224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.830336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.830448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.830584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.830697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.830818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.830931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.831045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.831156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.831268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.831379 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.831499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.831613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.831724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.831842 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.831954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.832058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.832169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.832282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.832393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.832505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.832615 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.832727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.832847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.832960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.833078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.833190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.833311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.833432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.833545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.833666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.833852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.833976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.834090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.834203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.834314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.834434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.834547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.834667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.834780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.834891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.835002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.835117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.835230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.835350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.835464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.835575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.835687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.835798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.835910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.836030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.836142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.836255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.836366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.836478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.836589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.836709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.836822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.836940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.837053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.837165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.837278 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.837398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.837512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.837624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80169) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 07:10:44.837750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 26 07:10:44.837750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80170) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 26 07:10:44.865331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.865601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.865688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.865840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.865919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.865994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.874288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.874421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.904332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.904544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.904668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.904778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.957732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:44.957971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.009831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.010095 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.010225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.010305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.074530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.074734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.074818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.074918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.074996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.075068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.075140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.075211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.075372 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.075510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.075592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.075665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.098490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.098708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.098852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.098942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.099090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.099173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.099320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.099411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.099540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.099621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.099757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.099837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.099972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.100192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.111157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.111472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.124078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.124296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.124440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.124563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.124644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.124706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.124826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.124951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.125029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.125143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.125219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.125359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80169) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.125483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.125613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80170) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 26 07:10:45.142190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.142401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 07:10:45.142738 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.142859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.143007 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 07:10:45.143129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 07:10:45.143251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.143448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.143570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.143695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 26 07:10:45.143789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.143895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.143973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.144079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.144364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.144621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.153554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.153857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.154126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.154341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.154465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.154613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.154878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.155159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.161265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.162136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.166138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.166422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.166701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.166977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.167099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.167226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.167315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.167505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.167629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.167764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.167857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.168001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.168090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.168287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.168411 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.168545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.168639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.168786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.168875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.169129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.169421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.169686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.191732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.192648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.193209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.193493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.194174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.194461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.198135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.198414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.198689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.199113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.199399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.199684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.205230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.205514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.206336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.206622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.209823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.210076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.210363 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.210643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.210860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.210984 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.211109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.211190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.211421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.211545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.211694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.211774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.211960 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.212251 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.212523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.212798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.227581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.227871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.228765 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.229048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.231802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.233366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.235325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.235618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.236522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.236844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.240067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.241288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.243285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.244498 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.246462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.246761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.247046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.247336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.247620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.247742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.247963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.248088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.248224 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.248526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.248813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.249113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.263868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.264158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.264425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.264686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.265126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.265423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.265688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.265977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.271756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.273044 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.273327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.273605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.274658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.274944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.276460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.276743 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.277034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.277312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.277600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.277788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.278092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.278386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.278667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.278958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.289022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.289295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.289571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.289870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.290149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.290426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.290548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.290686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.290803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.290952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.291253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.291530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.291803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.292335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.306280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.308584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.309218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.311567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.312191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.314565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.314804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.316347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.316630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.316908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.317207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.318929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.319220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.319509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.323382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.324966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.326009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.327602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.327907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.328193 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.328477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.328751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.330872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.330872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.331257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.331394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.338048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.340647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.340647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.341756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.342053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.342338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.346964 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80169) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.347241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.347530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.347813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.348087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.348366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.348639 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.348911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.349274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80170) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 26 07:10:45.349398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.349595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.349810 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.349908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.350090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.350215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.350322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.350507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.350687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.350815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80169) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.350940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.351065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.351256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.351627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.351916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.352219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.352484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.352760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.353020 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80170) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 26 07:10:45.384400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e2bd1889c0 pid: 80169 (default app) Jul 26 07:10:45.386744 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e2bd1889c0 pid: 80170 (default app) Jul 26 07:10:45.387853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b61a-0f21-42b3-b932-bd88534fcfbd None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:45.388134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b61a-0f21-42b3-b932-bd88534fcfbd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:45.388412 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86b61a-0f21-42b3-b932-bd88534fcfbd None None] PATH_INFO: `/v3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:45.392794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Tue Jul 26 07:10:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:10:53.010242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e58bd-73af-419f-ae54-8ee6b520054a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:53.010858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e58bd-73af-419f-ae54-8ee6b520054a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:53.010858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184e58bd-73af-419f-ae54-8ee6b520054a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:53.015304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:10:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:10:53.021201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:53.021666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:53.021780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:53.091945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: INFO dbcounter [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] Registered counter for database keystone Jul 26 07:10:53.115806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG oslo_db.sqlalchemy.engines [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80169) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 07:10:53.123462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writer thread running {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 07:10:53.306607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80169) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 07:10:53.307011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG passlib.handlers.bcrypt [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80169) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 07:10:53.362404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:10:53.364016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:10:53.498585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:10:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 07:10:53.643288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e58bd-73af-419f-ae54-8ee6b520054a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:53.713010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: INFO dbcounter [None req-184e58bd-73af-419f-ae54-8ee6b520054a None None] Registered counter for database keystone Jul 26 07:10:53.733265 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG oslo_db.sqlalchemy.engines [None req-184e58bd-73af-419f-ae54-8ee6b520054a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80170) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 26 07:10:53.740887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writer thread running {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 26 07:10:53.862347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184e58bd-73af-419f-ae54-8ee6b520054a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:53.867235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6575861-3a17-46c1-ba9b-822871cf1cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:53.867898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6575861-3a17-46c1-ba9b-822871cf1cae admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:53.868185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6575861-3a17-46c1-ba9b-822871cf1cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:53.868461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6575861-3a17-46c1-ba9b-822871cf1cae admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:53.877368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6575861-3a17-46c1-ba9b-822871cf1cae admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:53.929146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d6575861-3a17-46c1-ba9b-822871cf1cae admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 07:10:53.929335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6575861-3a17-46c1-ba9b-822871cf1cae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:53.940624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Tue Jul 26 07:10:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:10:53.946994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:53.962216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5044840-34f9-4dbb-9665-fb1631555f98 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:53.968026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117a441-afdb-467a-93b7-f3035d156408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:53.969180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117a441-afdb-467a-93b7-f3035d156408 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:53.969457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117a441-afdb-467a-93b7-f3035d156408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:53.969726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117a441-afdb-467a-93b7-f3035d156408 admin admin] PATH_INFO: `/v3/projects/9aaf3bcccded4ffcb2a432d5c86ce8ea` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:53.971586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117a441-afdb-467a-93b7-f3035d156408 admin admin] RBAC: Authorizing `identity:get_project(project_id=9aaf3bcccded4ffcb2a432d5c86ce8ea)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:54.022648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1117a441-afdb-467a-93b7-f3035d156408 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 26 07:10:54.023337 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117a441-afdb-467a-93b7-f3035d156408 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:54.027873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Tue Jul 26 07:10:53 2022] GET /identity/v3/projects/9aaf3bcccded4ffcb2a432d5c86ce8ea => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:10:56.013243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f09a3-5464-40a1-be3d-d8b7a03ae4b2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:56.013833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f09a3-5464-40a1-be3d-d8b7a03ae4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:56.013833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f09a3-5464-40a1-be3d-d8b7a03ae4b2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:56.018041 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:10:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:10:56.023512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02004dd-f1bf-4774-8374-7ecbbae0ece0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:56.023924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02004dd-f1bf-4774-8374-7ecbbae0ece0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:56.024057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02004dd-f1bf-4774-8374-7ecbbae0ece0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:56.047988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f02004dd-f1bf-4774-8374-7ecbbae0ece0 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:10:56.049592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f02004dd-f1bf-4774-8374-7ecbbae0ece0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:10:56.058032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:10:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 26 07:10:56.081208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f09a3-5464-40a1-be3d-d8b7a03ae4b2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:56.097383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f09a3-5464-40a1-be3d-d8b7a03ae4b2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:56.103273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef2c04-3318-4589-bc7a-e3ea80fa93f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:56.103900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef2c04-3318-4589-bc7a-e3ea80fa93f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:56.104171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef2c04-3318-4589-bc7a-e3ea80fa93f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:56.104445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef2c04-3318-4589-bc7a-e3ea80fa93f1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:56.113999 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef2c04-3318-4589-bc7a-e3ea80fa93f1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:56.115802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef2c04-3318-4589-bc7a-e3ea80fa93f1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:56.126193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Tue Jul 26 07:10:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:10:56.132409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02004dd-f1bf-4774-8374-7ecbbae0ece0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:56.147695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02004dd-f1bf-4774-8374-7ecbbae0ece0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:56.157067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac1ed8-bdf4-4ef2-98cd-bd086fd9219f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:56.158110 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac1ed8-bdf4-4ef2-98cd-bd086fd9219f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:56.158542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac1ed8-bdf4-4ef2-98cd-bd086fd9219f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:56.159001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac1ed8-bdf4-4ef2-98cd-bd086fd9219f admin admin] PATH_INFO: `/v3/users/9e725435b0294296a28451de9a0d08c4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:56.160549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac1ed8-bdf4-4ef2-98cd-bd086fd9219f admin admin] RBAC: Authorizing `identity:get_user(user_id=9e725435b0294296a28451de9a0d08c4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:56.162491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bac1ed8-bdf4-4ef2-98cd-bd086fd9219f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:56.164100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Tue Jul 26 07:10:56 2022] GET /identity/v3/users/9e725435b0294296a28451de9a0d08c4 => generated 247 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:10:58.225544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f94f46-35e1-4bf9-b68a-e89906131f3c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:58.226151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f94f46-35e1-4bf9-b68a-e89906131f3c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:58.226151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f94f46-35e1-4bf9-b68a-e89906131f3c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:58.227554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:10:58.233517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68501eb-9f7a-4040-bc3e-2036036cb2f0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:58.234056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68501eb-9f7a-4040-bc3e-2036036cb2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:58.234147 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68501eb-9f7a-4040-bc3e-2036036cb2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:58.255413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b68501eb-9f7a-4040-bc3e-2036036cb2f0 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:10:58.256646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b68501eb-9f7a-4040-bc3e-2036036cb2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:10:58.266944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f259-f9db-44d4-90d6-3afba9566106 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:58.267232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f259-f9db-44d4-90d6-3afba9566106 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:58.267544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f74f259-f9db-44d4-90d6-3afba9566106 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:58.268867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:10:58.274011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba721ecd-dfbe-464c-8825-7a19faf9881a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:58.274304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba721ecd-dfbe-464c-8825-7a19faf9881a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:58.274587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba721ecd-dfbe-464c-8825-7a19faf9881a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:58.316478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-ba721ecd-dfbe-464c-8825-7a19faf9881a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80170) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 26 07:10:58.316887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG passlib.handlers.bcrypt [None req-ba721ecd-dfbe-464c-8825-7a19faf9881a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80170) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 26 07:10:58.343161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 6/12] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:10:58.362877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ba721ecd-dfbe-464c-8825-7a19faf9881a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:10:58.364502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ba721ecd-dfbe-464c-8825-7a19faf9881a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:10:58.366658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68501eb-9f7a-4040-bc3e-2036036cb2f0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:58.391256 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:10:58.467840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 7/13] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:10:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:10:58.475224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68501eb-9f7a-4040-bc3e-2036036cb2f0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:58.490772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba721ecd-dfbe-464c-8825-7a19faf9881a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:58.512866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757fc8b-22b3-4f66-99a5-94f0c0d29e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:58.513536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757fc8b-22b3-4f66-99a5-94f0c0d29e17 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:58.513848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757fc8b-22b3-4f66-99a5-94f0c0d29e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:58.514131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757fc8b-22b3-4f66-99a5-94f0c0d29e17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:58.526093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8757fc8b-22b3-4f66-99a5-94f0c0d29e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8757fc8b-22b3-4f66-99a5-94f0c0d29e17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:10:58.535101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:10:58.536057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:10:58.536863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 7/14] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:10:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:10:58.542480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757fc8b-22b3-4f66-99a5-94f0c0d29e17 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:58.593933 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba721ecd-dfbe-464c-8825-7a19faf9881a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:58.596847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:10:58.623528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:10:58.634372 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe329607-4e07-4f76-9469-e3351f6be369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:58.635286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe329607-4e07-4f76-9469-e3351f6be369 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:58.635888 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe329607-4e07-4f76-9469-e3351f6be369 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:58.636166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe329607-4e07-4f76-9469-e3351f6be369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:58.642815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8757fc8b-22b3-4f66-99a5-94f0c0d29e17 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:58.678799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4e264-499e-4b2f-b5c2-b1f880cf437a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:58.679481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4e264-499e-4b2f-b5c2-b1f880cf437a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:58.679763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4e264-499e-4b2f-b5c2-b1f880cf437a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:58.680031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4e264-499e-4b2f-b5c2-b1f880cf437a None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:58.680779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4e264-499e-4b2f-b5c2-b1f880cf437a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:10:58.681086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4e264-499e-4b2f-b5c2-b1f880cf437a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:58.682541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4e264-499e-4b2f-b5c2-b1f880cf437a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:58.695430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 8/15] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:10:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:10:58.701412 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4e264-499e-4b2f-b5c2-b1f880cf437a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:58.709059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe329607-4e07-4f76-9469-e3351f6be369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:58.710644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe329607-4e07-4f76-9469-e3351f6be369 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:58.802328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4e264-499e-4b2f-b5c2-b1f880cf437a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:58.809320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 8/16] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:10:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 07:10:58.815702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe329607-4e07-4f76-9469-e3351f6be369 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:58.819059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:10:58.928601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe329607-4e07-4f76-9469-e3351f6be369 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:58.949771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80c91-1337-463a-a86a-513d48aed182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:58.950439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80c91-1337-463a-a86a-513d48aed182 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:58.950723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80c91-1337-463a-a86a-513d48aed182 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:58.951017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff80c91-1337-463a-a86a-513d48aed182 None admin] PATH_INFO: `/v3/users/9e725435b0294296a28451de9a0d08c4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:58.952200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff80c91-1337-463a-a86a-513d48aed182 None admin] RBAC: Authorizing `identity:get_user(user_id=9e725435b0294296a28451de9a0d08c4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:58.953650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff80c91-1337-463a-a86a-513d48aed182 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:58.955286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 9/17] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:10:58 2022] GET /identity/v3/users/9e725435b0294296a28451de9a0d08c4 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:10:58.960997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80c91-1337-463a-a86a-513d48aed182 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:58.968844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8e8a7-b08d-4bbd-b8cc-03892325cf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:58.970200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8e8a7-b08d-4bbd-b8cc-03892325cf45 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:58.970484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8e8a7-b08d-4bbd-b8cc-03892325cf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:58.970758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8e8a7-b08d-4bbd-b8cc-03892325cf45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:58.976004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:10:58.990841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8e8a7-b08d-4bbd-b8cc-03892325cf45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:58.992326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8e8a7-b08d-4bbd-b8cc-03892325cf45 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eec8e8a7-b08d-4bbd-b8cc-03892325cf45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:10:59.012743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:10:59.013828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:10:59.014625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:10:59.014625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:10:59.014625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:10:59.014625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:10:59.014625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 9/18] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:10:59.020114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8e8a7-b08d-4bbd-b8cc-03892325cf45 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:59.064493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff80c91-1337-463a-a86a-513d48aed182 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:59.075864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:10:59.098316 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86871da3-d84b-40c5-b4b0-8236e5143f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:59.098978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86871da3-d84b-40c5-b4b0-8236e5143f32 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:59.099260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86871da3-d84b-40c5-b4b0-8236e5143f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:59.099535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86871da3-d84b-40c5-b4b0-8236e5143f32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:59.100753 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86871da3-d84b-40c5-b4b0-8236e5143f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:59.102216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86871da3-d84b-40c5-b4b0-8236e5143f32 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:59.104028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 10/19] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:10:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:10:59.110206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86871da3-d84b-40c5-b4b0-8236e5143f32 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:10:59.128971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8e8a7-b08d-4bbd-b8cc-03892325cf45 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:59.162189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50719ad-310b-49ff-84ff-e846d54c42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:59.162817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50719ad-310b-49ff-84ff-e846d54c42ff None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:59.163100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50719ad-310b-49ff-84ff-e846d54c42ff None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:59.163381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50719ad-310b-49ff-84ff-e846d54c42ff None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:59.164082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50719ad-310b-49ff-84ff-e846d54c42ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:10:59.164381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50719ad-310b-49ff-84ff-e846d54c42ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:59.165800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50719ad-310b-49ff-84ff-e846d54c42ff None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:59.183614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 10/20] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:10:59.192019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:10:59.214922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86871da3-d84b-40c5-b4b0-8236e5143f32 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:10:59.254634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5c5d1-c87e-423a-89b4-a00a429bbdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:10:59.255349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c5d1-c87e-423a-89b4-a00a429bbdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:10:59.255520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c5d1-c87e-423a-89b4-a00a429bbdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:10:59.255834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5c5d1-c87e-423a-89b4-a00a429bbdc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:10:59.256656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5c5d1-c87e-423a-89b4-a00a429bbdc7 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.domain.id=default, user.id=9e725435b0294296a28451de9a0d08c4) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:10:59.256982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5c5d1-c87e-423a-89b4-a00a429bbdc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:10:59.258490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5c5d1-c87e-423a-89b4-a00a429bbdc7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:10:59.266440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 11/21] 10.222.0.39 () {56 vars in 1322 bytes} [Tue Jul 26 07:10:59 2022] GET /identity/v3/role_assignments?user.id=9e725435b0294296a28451de9a0d08c4&scope.domain.id=default&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 233 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:01.596849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbf1df-6174-435e-a8f3-f7d5bc0d51e8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:01.597359 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbf1df-6174-435e-a8f3-f7d5bc0d51e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:01.597460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fbf1df-6174-435e-a8f3-f7d5bc0d51e8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:01.599177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 11/22] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:01.604534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae28986-cd64-4ed9-8260-c0c372a8d39a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:01.605010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae28986-cd64-4ed9-8260-c0c372a8d39a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:01.605082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae28986-cd64-4ed9-8260-c0c372a8d39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:01.629098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0ae28986-cd64-4ed9-8260-c0c372a8d39a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:01.630360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0ae28986-cd64-4ed9-8260-c0c372a8d39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:01.730396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 12/23] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:01.765249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fc3b6-25a8-4984-a5de-326524f273cd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:01.765928 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fc3b6-25a8-4984-a5de-326524f273cd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:01.766119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae28986-cd64-4ed9-8260-c0c372a8d39a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:01.766594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fc3b6-25a8-4984-a5de-326524f273cd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:01.768218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 12/24] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:01.775318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b079-1488-49e1-9883-d87db0d12372 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:01.775910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b079-1488-49e1-9883-d87db0d12372 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:01.776397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43b079-1488-49e1-9883-d87db0d12372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:01.815108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8e43b079-1488-49e1-9883-d87db0d12372 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:01.817191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8e43b079-1488-49e1-9883-d87db0d12372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:01.923848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:01.937324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 13/25] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:01.940024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cc48e-a843-451e-afa0-9ebe4da7f167 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:01.940325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cc48e-a843-451e-afa0-9ebe4da7f167 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:01.940602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cc48e-a843-451e-afa0-9ebe4da7f167 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:01.941927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 14/26] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:01.951525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07ae1b-b3ca-4215-a2a3-12097a42507b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:01.951816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07ae1b-b3ca-4215-a2a3-12097a42507b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:01.952102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d07ae1b-b3ca-4215-a2a3-12097a42507b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:01.970474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae28986-cd64-4ed9-8260-c0c372a8d39a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:01.978366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7d07ae1b-b3ca-4215-a2a3-12097a42507b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:01.979866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d07ae1b-b3ca-4215-a2a3-12097a42507b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:02.037196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0721027c-7b96-4877-a30f-7b3b4fc76e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:02.037866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721027c-7b96-4877-a30f-7b3b4fc76e98 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:02.038149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721027c-7b96-4877-a30f-7b3b4fc76e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:02.038421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0721027c-7b96-4877-a30f-7b3b4fc76e98 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:02.039153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721027c-7b96-4877-a30f-7b3b4fc76e98 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:02.040559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0721027c-7b96-4877-a30f-7b3b4fc76e98 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:02.052528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:02.083188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 15/27] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:02.084412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07ae1b-b3ca-4215-a2a3-12097a42507b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:02.176313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 13/28] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Jul 26 07:11:01 2022] POST /identity/v3/roles => generated 219 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:11:02.178156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d0ffd-d43f-4694-9b8d-03020dc0ed3b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:02.178748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d0ffd-d43f-4694-9b8d-03020dc0ed3b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:02.179243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d0ffd-d43f-4694-9b8d-03020dc0ed3b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:02.180902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 14/29] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:02.181927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d0ffd-d43f-4694-9b8d-03020dc0ed3b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:02.212412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d07ae1b-b3ca-4215-a2a3-12097a42507b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:02.299519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089f343-a649-4343-bee8-91f3a3fe2416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:02.300117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089f343-a649-4343-bee8-91f3a3fe2416 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:02.300228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d0ffd-d43f-4694-9b8d-03020dc0ed3b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:02.300409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089f343-a649-4343-bee8-91f3a3fe2416 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:02.300702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2089f343-a649-4343-bee8-91f3a3fe2416 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:02.301480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089f343-a649-4343-bee8-91f3a3fe2416 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:02.302957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2089f343-a649-4343-bee8-91f3a3fe2416 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:02.341271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751d8f4-802f-4c88-a884-d70cafbd140b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:02.342401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d8f4-802f-4c88-a884-d70cafbd140b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:02.342880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d8f4-802f-4c88-a884-d70cafbd140b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:02.343373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751d8f4-802f-4c88-a884-d70cafbd140b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:02.356339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751d8f4-802f-4c88-a884-d70cafbd140b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:02.364801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0751d8f4-802f-4c88-a884-d70cafbd140b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:02.366527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:02.367346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:02.368543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 15/30] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:11:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:02.370470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3fcda-00a5-455b-88db-203016314d0b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:02.372207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3fcda-00a5-455b-88db-203016314d0b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:02.372540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3fcda-00a5-455b-88db-203016314d0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:02.400982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8cd3fcda-00a5-455b-88db-203016314d0b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:02.402765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8cd3fcda-00a5-455b-88db-203016314d0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:02.487619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 16/31] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:02.488515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3fcda-00a5-455b-88db-203016314d0b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:02.596188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3fcda-00a5-455b-88db-203016314d0b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:02.621929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:02.633726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d9cdc-a4e3-4617-a79f-f6c9740c6ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:02.634668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d9cdc-a4e3-4617-a79f-f6c9740c6ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:02.634951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d9cdc-a4e3-4617-a79f-f6c9740c6ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:02.635226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d9cdc-a4e3-4617-a79f-f6c9740c6ea6 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:02.635965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d9cdc-a4e3-4617-a79f-f6c9740c6ea6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:02.636273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d9cdc-a4e3-4617-a79f-f6c9740c6ea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:02.637854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d9cdc-a4e3-4617-a79f-f6c9740c6ea6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:02.643104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 16/32] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Jul 26 07:11:02 2022] POST /identity/v3/roles => generated 225 bytes in 559 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:11:02.644749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089f343-a649-4343-bee8-91f3a3fe2416 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:02.651556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 17/33] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:02.662920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d9cdc-a4e3-4617-a79f-f6c9740c6ea6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:02.726324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:02.821624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d9cdc-a4e3-4617-a79f-f6c9740c6ea6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:02.860284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f75feb-9a54-4a5e-a946-320d2618ca90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:02.861342 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f75feb-9a54-4a5e-a946-320d2618ca90 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:02.861809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f75feb-9a54-4a5e-a946-320d2618ca90 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:02.862282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f75feb-9a54-4a5e-a946-320d2618ca90 None admin] PATH_INFO: `/v3/users/9e725435b0294296a28451de9a0d08c4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:02.863731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f75feb-9a54-4a5e-a946-320d2618ca90 None admin] RBAC: Authorizing `identity:get_user(user_id=9e725435b0294296a28451de9a0d08c4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:02.865393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f75feb-9a54-4a5e-a946-320d2618ca90 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:02.867285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 18/34] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:02 2022] GET /identity/v3/users/9e725435b0294296a28451de9a0d08c4 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:02.872886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f75feb-9a54-4a5e-a946-320d2618ca90 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:02.893297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2089f343-a649-4343-bee8-91f3a3fe2416 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:02.939797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c967c-2c88-409c-b8ee-72269573ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:02.940984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c967c-2c88-409c-b8ee-72269573ea70 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:02.941460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c967c-2c88-409c-b8ee-72269573ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:02.941949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1c967c-2c88-409c-b8ee-72269573ea70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:02.954802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c967c-2c88-409c-b8ee-72269573ea70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:02.956787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1c967c-2c88-409c-b8ee-72269573ea70 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:02.958045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0b1c967c-2c88-409c-b8ee-72269573ea70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:11:02.974391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:02.975306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:02.976091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:11:02.976091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:11:02.976091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:11:02.976091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:02.977887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 17/35] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:02.987216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c967c-2c88-409c-b8ee-72269573ea70 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:03.006946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f75feb-9a54-4a5e-a946-320d2618ca90 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:03.055574 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861a683-3c32-474a-a545-e20c924cc994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:03.056299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861a683-3c32-474a-a545-e20c924cc994 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.056584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861a683-3c32-474a-a545-e20c924cc994 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.056858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861a683-3c32-474a-a545-e20c924cc994 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.058093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861a683-3c32-474a-a545-e20c924cc994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:03.059689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861a683-3c32-474a-a545-e20c924cc994 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:03.061595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 19/36] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:03.067196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861a683-3c32-474a-a545-e20c924cc994 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:03.122525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:03.135300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1c967c-2c88-409c-b8ee-72269573ea70 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:03.171009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abeee00-d996-4a4e-9881-46faa99287c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:03.171661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abeee00-d996-4a4e-9881-46faa99287c0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.171944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abeee00-d996-4a4e-9881-46faa99287c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.172214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abeee00-d996-4a4e-9881-46faa99287c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.172921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abeee00-d996-4a4e-9881-46faa99287c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:03.173225 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abeee00-d996-4a4e-9881-46faa99287c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:03.174854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abeee00-d996-4a4e-9881-46faa99287c0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:03.181473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861a683-3c32-474a-a545-e20c924cc994 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:03.191643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 18/37] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:03.191643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcf0ad-1934-4cdb-96be-7b2cd1f9dd3f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.191643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcf0ad-1934-4cdb-96be-7b2cd1f9dd3f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.191643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcf0ad-1934-4cdb-96be-7b2cd1f9dd3f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.191643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 19/38] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:03.194897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcf0ad-1934-4cdb-96be-7b2cd1f9dd3f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:03.209483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:03.221574 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02c951e-de56-47cc-8325-e2efb8d049ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:03.222301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c951e-de56-47cc-8325-e2efb8d049ef None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.222588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c951e-de56-47cc-8325-e2efb8d049ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.222872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02c951e-de56-47cc-8325-e2efb8d049ef None admin] PATH_INFO: `/v3/domains/default/users/9e725435b0294296a28451de9a0d08c4/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.224898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c951e-de56-47cc-8325-e2efb8d049ef None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9e725435b0294296a28451de9a0d08c4, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:03.226418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02c951e-de56-47cc-8325-e2efb8d049ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:03.244503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 20/39] 10.222.0.39 () {58 vars in 1361 bytes} [Tue Jul 26 07:11:03 2022] PUT /identity/v3/domains/default/users/9e725435b0294296a28451de9a0d08c4/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:11:03.245813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfa44c-2d92-48ba-8c60-2b1f9f61b353 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.246109 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfa44c-2d92-48ba-8c60-2b1f9f61b353 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.246385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbfa44c-2d92-48ba-8c60-2b1f9f61b353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.275127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5fbfa44c-2d92-48ba-8c60-2b1f9f61b353 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:03.276396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5fbfa44c-2d92-48ba-8c60-2b1f9f61b353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:03.338523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcf0ad-1934-4cdb-96be-7b2cd1f9dd3f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:03.366940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 21/40] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:03.368802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d0268-c7b6-40df-b170-554ff9091e71 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.369453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d0268-c7b6-40df-b170-554ff9091e71 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.369875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d0268-c7b6-40df-b170-554ff9091e71 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.371452 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 22/41] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:03.376893 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3f5d7-b173-4130-b026-74d52b05ca2d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.377126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3f5d7-b173-4130-b026-74d52b05ca2d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.377411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf3f5d7-b173-4130-b026-74d52b05ca2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.382649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6f3a2-baba-4c9c-9c58-f76ed380c2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:03.383318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6f3a2-baba-4c9c-9c58-f76ed380c2ed None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.383600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6f3a2-baba-4c9c-9c58-f76ed380c2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.383876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb6f3a2-baba-4c9c-9c58-f76ed380c2ed None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.388846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6f3a2-baba-4c9c-9c58-f76ed380c2ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:03.390442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb6f3a2-baba-4c9c-9c58-f76ed380c2ed None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:03.405350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9cf3f5d7-b173-4130-b026-74d52b05ca2d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:03.406635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9cf3f5d7-b173-4130-b026-74d52b05ca2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:03.427888 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:03.436787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 20/42] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:11:03 2022] POST /identity/v3/projects => generated 301 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:11:03.437731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6f3a2-baba-4c9c-9c58-f76ed380c2ed None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:03.511565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 23/43] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:03.534090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3f5d7-b173-4130-b026-74d52b05ca2d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:03.587213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb6f3a2-baba-4c9c-9c58-f76ed380c2ed None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:03.601183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:03.635149 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072ac08-bcfe-43d0-aa79-58fa23664c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:03.635724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072ac08-bcfe-43d0-aa79-58fa23664c0b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.636004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072ac08-bcfe-43d0-aa79-58fa23664c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.636274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072ac08-bcfe-43d0-aa79-58fa23664c0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.638058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072ac08-bcfe-43d0-aa79-58fa23664c0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:03.639674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072ac08-bcfe-43d0-aa79-58fa23664c0b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:03.641450 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 21/44] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:03.647148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072ac08-bcfe-43d0-aa79-58fa23664c0b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:03.677467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf3f5d7-b173-4130-b026-74d52b05ca2d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:03.718280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cce33e-4a9d-4918-a49d-32031874b454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:03.719425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cce33e-4a9d-4918-a49d-32031874b454 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.719713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cce33e-4a9d-4918-a49d-32031874b454 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.719992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cce33e-4a9d-4918-a49d-32031874b454 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.720734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cce33e-4a9d-4918-a49d-32031874b454 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:03.722242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cce33e-4a9d-4918-a49d-32031874b454 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:03.723995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:03.741901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 24/45] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Jul 26 07:11:03 2022] POST /identity/v3/roles => generated 223 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:11:03.764576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072ac08-bcfe-43d0-aa79-58fa23664c0b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:03.807892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f489fe85-db6f-4044-a5af-a7141ecf92b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:03.808554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489fe85-db6f-4044-a5af-a7141ecf92b7 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:03.808831 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489fe85-db6f-4044-a5af-a7141ecf92b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:03.809103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f489fe85-db6f-4044-a5af-a7141ecf92b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:03.809857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489fe85-db6f-4044-a5af-a7141ecf92b7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:03.811420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f489fe85-db6f-4044-a5af-a7141ecf92b7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:03.847142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 22/46] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:11:03 2022] POST /identity/v3/projects => generated 312 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:11:05.303421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbee35f-ca3e-4eda-8cec-69c19356c9ac None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:05.303967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbee35f-ca3e-4eda-8cec-69c19356c9ac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:05.303967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbee35f-ca3e-4eda-8cec-69c19356c9ac None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:05.305336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 25/47] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:05.311013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495021bc-c2f9-4ad4-9e49-327fb584f71c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:05.311510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495021bc-c2f9-4ad4-9e49-327fb584f71c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:05.311510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495021bc-c2f9-4ad4-9e49-327fb584f71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:05.331263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-495021bc-c2f9-4ad4-9e49-327fb584f71c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:05.332792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-495021bc-c2f9-4ad4-9e49-327fb584f71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:05.404504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 23/48] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:05.428221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbee35f-ca3e-4eda-8cec-69c19356c9ac None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:05.543911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbee35f-ca3e-4eda-8cec-69c19356c9ac None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:05.545518 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:05.580844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7968ab2-5c41-4d60-988d-15b7f7008de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:05.581370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7968ab2-5c41-4d60-988d-15b7f7008de1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:05.581656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7968ab2-5c41-4d60-988d-15b7f7008de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:05.581966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7968ab2-5c41-4d60-988d-15b7f7008de1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:05.592588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7968ab2-5c41-4d60-988d-15b7f7008de1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d7968ab2-5c41-4d60-988d-15b7f7008de1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:05.597660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:05.598580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:05.599388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 26/49] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:11:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:05.605943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495021bc-c2f9-4ad4-9e49-327fb584f71c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:05.668590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:05.720068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495021bc-c2f9-4ad4-9e49-327fb584f71c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:05.756542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae549f3-7060-41af-be69-682590ab7253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:05.757278 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae549f3-7060-41af-be69-682590ab7253 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:05.757631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae549f3-7060-41af-be69-682590ab7253 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:05.757986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae549f3-7060-41af-be69-682590ab7253 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:05.758792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae549f3-7060-41af-be69-682590ab7253 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:05.759175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae549f3-7060-41af-be69-682590ab7253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:05.760658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae549f3-7060-41af-be69-682590ab7253 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:05.773845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 24/50] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:05.779713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7968ab2-5c41-4d60-988d-15b7f7008de1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:05.843679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b51190-c962-4962-8074-085899613d2a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:05.843913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b51190-c962-4962-8074-085899613d2a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:05.844200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b51190-c962-4962-8074-085899613d2a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:05.845600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 25/51] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:05.850909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972630fc-5a10-40a1-9fab-a3b6fc762a79 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:05.851205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972630fc-5a10-40a1-9fab-a3b6fc762a79 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:05.851478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972630fc-5a10-40a1-9fab-a3b6fc762a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:05.871649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-972630fc-5a10-40a1-9fab-a3b6fc762a79 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:05.872868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-972630fc-5a10-40a1-9fab-a3b6fc762a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:05.882679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7968ab2-5c41-4d60-988d-15b7f7008de1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:05.925512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846e890-0ad5-4eb0-a8a8-0c8d1e154416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:05.926204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846e890-0ad5-4eb0-a8a8-0c8d1e154416 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:05.926483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846e890-0ad5-4eb0-a8a8-0c8d1e154416 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:05.927351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846e890-0ad5-4eb0-a8a8-0c8d1e154416 None admin] PATH_INFO: `/v3/users/9e725435b0294296a28451de9a0d08c4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:05.928528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846e890-0ad5-4eb0-a8a8-0c8d1e154416 None admin] RBAC: Authorizing `identity:get_user(user_id=9e725435b0294296a28451de9a0d08c4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:05.930133 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846e890-0ad5-4eb0-a8a8-0c8d1e154416 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:05.931823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 27/52] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:05 2022] GET /identity/v3/users/9e725435b0294296a28451de9a0d08c4 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:05.937518 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846e890-0ad5-4eb0-a8a8-0c8d1e154416 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:05.952458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 26/53] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:05.968629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:05.976491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972630fc-5a10-40a1-9fab-a3b6fc762a79 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:06.020884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:06.040650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846e890-0ad5-4eb0-a8a8-0c8d1e154416 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:06.076230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13c91f-f677-4c9d-a75d-3114eae88a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:06.076899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13c91f-f677-4c9d-a75d-3114eae88a0d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:06.077181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13c91f-f677-4c9d-a75d-3114eae88a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:06.077455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f13c91f-f677-4c9d-a75d-3114eae88a0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:06.078672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13c91f-f677-4c9d-a75d-3114eae88a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:06.079165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972630fc-5a10-40a1-9fab-a3b6fc762a79 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:06.080152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f13c91f-f677-4c9d-a75d-3114eae88a0d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:06.081948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 28/54] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:06.087974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13c91f-f677-4c9d-a75d-3114eae88a0d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:06.116160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe07ae-8798-448d-a23e-6a98b8f06915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:06.116736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe07ae-8798-448d-a23e-6a98b8f06915 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:06.117125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe07ae-8798-448d-a23e-6a98b8f06915 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:06.117301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe07ae-8798-448d-a23e-6a98b8f06915 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:06.118645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe07ae-8798-448d-a23e-6a98b8f06915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:06.120106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe07ae-8798-448d-a23e-6a98b8f06915 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:06.121827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 27/55] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:06.127817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe07ae-8798-448d-a23e-6a98b8f06915 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:06.191050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:06.195801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f13c91f-f677-4c9d-a75d-3114eae88a0d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:06.228062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe07ae-8798-448d-a23e-6a98b8f06915 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:06.230393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de362ba-e951-4557-9627-37a57f588967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:06.231036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de362ba-e951-4557-9627-37a57f588967 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:06.231321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de362ba-e951-4557-9627-37a57f588967 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:06.231598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de362ba-e951-4557-9627-37a57f588967 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:06.232396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de362ba-e951-4557-9627-37a57f588967 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.domain.id=default, user.id=9e725435b0294296a28451de9a0d08c4) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:06.232709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de362ba-e951-4557-9627-37a57f588967 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:06.234582 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de362ba-e951-4557-9627-37a57f588967 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:06.239964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:06.242307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 29/56] 10.222.0.39 () {56 vars in 1322 bytes} [Tue Jul 26 07:11:06 2022] GET /identity/v3/role_assignments?user.id=9e725435b0294296a28451de9a0d08c4&scope.domain.id=default&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:06.266602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dee89b8-42ec-4890-8a85-886f4b7f2858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:06.267260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee89b8-42ec-4890-8a85-886f4b7f2858 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:06.267545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee89b8-42ec-4890-8a85-886f4b7f2858 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:06.267818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dee89b8-42ec-4890-8a85-886f4b7f2858 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:06.268647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee89b8-42ec-4890-8a85-886f4b7f2858 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:06.270207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dee89b8-42ec-4890-8a85-886f4b7f2858 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:06.301579 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 28/57] 10.222.0.39 () {60 vars in 1140 bytes} [Tue Jul 26 07:11:06 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:11:08.312338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987625a7-7555-4683-849b-eb2dd3bd6338 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:08.312888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987625a7-7555-4683-849b-eb2dd3bd6338 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:08.312888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987625a7-7555-4683-849b-eb2dd3bd6338 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:08.314306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 30/58] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:08.319897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78f1d9-8ae9-47da-ada4-f5ae43ea3aea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:08.320331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78f1d9-8ae9-47da-ada4-f5ae43ea3aea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:08.320415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78f1d9-8ae9-47da-ada4-f5ae43ea3aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:08.339419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2e78f1d9-8ae9-47da-ada4-f5ae43ea3aea None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:08.340620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2e78f1d9-8ae9-47da-ada4-f5ae43ea3aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:08.411558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 29/59] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:08.436026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987625a7-7555-4683-849b-eb2dd3bd6338 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:08.542246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987625a7-7555-4683-849b-eb2dd3bd6338 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:08.575357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2136b611-21ab-4c47-bd50-adb28a1261c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:08.575966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136b611-21ab-4c47-bd50-adb28a1261c4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:08.576243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136b611-21ab-4c47-bd50-adb28a1261c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:08.576517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2136b611-21ab-4c47-bd50-adb28a1261c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:08.577656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136b611-21ab-4c47-bd50-adb28a1261c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:08.579155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2136b611-21ab-4c47-bd50-adb28a1261c4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:08.580797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 31/60] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:08.586654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78f1d9-8ae9-47da-ada4-f5ae43ea3aea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:08.679551 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:08.689505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78f1d9-8ae9-47da-ada4-f5ae43ea3aea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:08.723513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f9e7f-6566-4aa5-86d2-b3aba1f4b648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:08.724140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f9e7f-6566-4aa5-86d2-b3aba1f4b648 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:08.724420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f9e7f-6566-4aa5-86d2-b3aba1f4b648 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:08.724693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f9e7f-6566-4aa5-86d2-b3aba1f4b648 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:08.725387 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f9e7f-6566-4aa5-86d2-b3aba1f4b648 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:08.726849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f9e7f-6566-4aa5-86d2-b3aba1f4b648 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:08.754872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 30/61] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:11:08 2022] POST /identity/v3/users => generated 275 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:11:11.721861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd0805-4f6e-47ab-b182-b42b58937667 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:11.722864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd0805-4f6e-47ab-b182-b42b58937667 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:11.723349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcd0805-4f6e-47ab-b182-b42b58937667 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:11.724930 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 32/62] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:11.730704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12183126-d5ad-4b0d-beff-4bb6be2ca156 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:11.731593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12183126-d5ad-4b0d-beff-4bb6be2ca156 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:11.732155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12183126-d5ad-4b0d-beff-4bb6be2ca156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:11.755903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-12183126-d5ad-4b0d-beff-4bb6be2ca156 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:11.757503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-12183126-d5ad-4b0d-beff-4bb6be2ca156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:11.856384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 31/63] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:11.903223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd0805-4f6e-47ab-b182-b42b58937667 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:11.941121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:12.043593 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcd0805-4f6e-47ab-b182-b42b58937667 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:12.073560 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27243522-cbae-43f3-9a1a-6a7aaaa547a5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.074224 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27243522-cbae-43f3-9a1a-6a7aaaa547a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.074698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27243522-cbae-43f3-9a1a-6a7aaaa547a5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.076563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 32/64] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:12.082958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47132db-7a87-431c-88f5-8d42d7ea2d04 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.083506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47132db-7a87-431c-88f5-8d42d7ea2d04 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.084134 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47132db-7a87-431c-88f5-8d42d7ea2d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.092831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59de8311-ef75-4aff-be3c-4feacaab3290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:12.093502 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de8311-ef75-4aff-be3c-4feacaab3290 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.101761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de8311-ef75-4aff-be3c-4feacaab3290 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.101761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59de8311-ef75-4aff-be3c-4feacaab3290 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.104461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b47132db-7a87-431c-88f5-8d42d7ea2d04 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:12.104844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59de8311-ef75-4aff-be3c-4feacaab3290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:12.106204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b47132db-7a87-431c-88f5-8d42d7ea2d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-59de8311-ef75-4aff-be3c-4feacaab3290 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:12.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:12.112167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:12.112916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 33/65] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:12.113751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8287f6-81b6-4f90-84dd-72173b589e4a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.114227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8287f6-81b6-4f90-84dd-72173b589e4a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.119349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8287f6-81b6-4f90-84dd-72173b589e4a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.120900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 34/66] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:12.123326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8287f6-81b6-4f90-84dd-72173b589e4a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:12.180181 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 33/67] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:12.181727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a9884-edd6-40b0-83c5-d0726409ac58 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.182254 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a9884-edd6-40b0-83c5-d0726409ac58 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.182716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686a9884-edd6-40b0-83c5-d0726409ac58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.201819 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-686a9884-edd6-40b0-83c5-d0726409ac58 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:12.203373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-686a9884-edd6-40b0-83c5-d0726409ac58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:12.214117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:12.220574 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:12.231411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8287f6-81b6-4f90-84dd-72173b589e4a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:12.266882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b2e0b-1fa9-4ebb-b3b0-0346d81db82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:12.267798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b2e0b-1fa9-4ebb-b3b0-0346d81db82b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.268280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b2e0b-1fa9-4ebb-b3b0-0346d81db82b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.268698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b2e0b-1fa9-4ebb-b3b0-0346d81db82b None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.269566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b2e0b-1fa9-4ebb-b3b0-0346d81db82b None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:12.270036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b2e0b-1fa9-4ebb-b3b0-0346d81db82b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:12.271680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b2e0b-1fa9-4ebb-b3b0-0346d81db82b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:12.279016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 34/68] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:12.280157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a9884-edd6-40b0-83c5-d0726409ac58 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:12.281496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 35/69] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:12.287649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b2e0b-1fa9-4ebb-b3b0-0346d81db82b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:12.394655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686a9884-edd6-40b0-83c5-d0726409ac58 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:12.394953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b2e0b-1fa9-4ebb-b3b0-0346d81db82b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:12.432110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ac9b8-c1ac-4faf-869a-1a6182e2cfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:12.432386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb68b36-1f04-4842-818d-27b7f451bf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:12.432771 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ac9b8-c1ac-4faf-869a-1a6182e2cfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.433054 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ac9b8-c1ac-4faf-869a-1a6182e2cfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.433246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb68b36-1f04-4842-818d-27b7f451bf2e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.433338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ac9b8-c1ac-4faf-869a-1a6182e2cfe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.433530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb68b36-1f04-4842-818d-27b7f451bf2e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.433835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb68b36-1f04-4842-818d-27b7f451bf2e None admin] PATH_INFO: `/v3/users/5c36c5fbc8c04f3f8235a61fc5992850` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.444413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:12.449042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ac9b8-c1ac-4faf-869a-1a6182e2cfe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:12.451050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb68b36-1f04-4842-818d-27b7f451bf2e None admin] RBAC: Authorizing `identity:get_user(user_id=5c36c5fbc8c04f3f8235a61fc5992850)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:12.452603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb68b36-1f04-4842-818d-27b7f451bf2e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:12.454366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 36/70] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:12 2022] GET /identity/v3/users/5c36c5fbc8c04f3f8235a61fc5992850 => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:12.459149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb68b36-1f04-4842-818d-27b7f451bf2e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7b9ac9b8-c1ac-4faf-869a-1a6182e2cfe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:12.459221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:12.460023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 35/71] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:11:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:12.460754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b9dde-a810-40dc-854e-9e97879ea2c8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.461409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b9dde-a810-40dc-854e-9e97879ea2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.461409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b9dde-a810-40dc-854e-9e97879ea2c8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.461409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 36/72] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:12.466680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b9dde-a810-40dc-854e-9e97879ea2c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:12.477718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:12.571587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb68b36-1f04-4842-818d-27b7f451bf2e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:12.573638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b9dde-a810-40dc-854e-9e97879ea2c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:12.607673 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8996b05e-6ffe-472c-9c9e-5b0eada0d4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:12.608804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b05e-6ffe-472c-9c9e-5b0eada0d4ce None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.609162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b05e-6ffe-472c-9c9e-5b0eada0d4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.609508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8996b05e-6ffe-472c-9c9e-5b0eada0d4ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.609785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effea7d-4dbc-4201-8873-67c904c16619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:12.610445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effea7d-4dbc-4201-8873-67c904c16619 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.610728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effea7d-4dbc-4201-8873-67c904c16619 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.611001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0effea7d-4dbc-4201-8873-67c904c16619 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.611962 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effea7d-4dbc-4201-8873-67c904c16619 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:12.612610 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effea7d-4dbc-4201-8873-67c904c16619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:12.614045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0effea7d-4dbc-4201-8873-67c904c16619 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:12.620422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8996b05e-6ffe-472c-9c9e-5b0eada0d4ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:12.623362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 37/73] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8996b05e-6ffe-472c-9c9e-5b0eada0d4ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:12.625657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:12.626607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:12.632477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 37/74] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:12.633098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effea7d-4dbc-4201-8873-67c904c16619 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:12.633165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e429cf-36de-43a1-833c-03cc2c1a7fa4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.633165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e429cf-36de-43a1-833c-03cc2c1a7fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.633165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e429cf-36de-43a1-833c-03cc2c1a7fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.650379 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-59e429cf-36de-43a1-833c-03cc2c1a7fa4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:12.651511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-59e429cf-36de-43a1-833c-03cc2c1a7fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:12.693873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:12.726040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:12.732180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 38/75] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:12.733441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e429cf-36de-43a1-833c-03cc2c1a7fa4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:12.755935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0effea7d-4dbc-4201-8873-67c904c16619 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:12.793196 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f11f-5f13-48b6-ab64-c255434dd6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:12.793859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f11f-5f13-48b6-ab64-c255434dd6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.794205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f11f-5f13-48b6-ab64-c255434dd6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.794487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f11f-5f13-48b6-ab64-c255434dd6a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.847172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e429cf-36de-43a1-833c-03cc2c1a7fa4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:12.863791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f11f-5f13-48b6-ab64-c255434dd6a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:12.865366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f11f-5f13-48b6-ab64-c255434dd6a4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:12.881615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c474072-39df-4349-989f-bbc8cc018162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:12.882433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c474072-39df-4349-989f-bbc8cc018162 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.882714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c474072-39df-4349-989f-bbc8cc018162 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.883098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c474072-39df-4349-989f-bbc8cc018162 None admin] PATH_INFO: `/v3/users/9e725435b0294296a28451de9a0d08c4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.884282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c474072-39df-4349-989f-bbc8cc018162 None admin] RBAC: Authorizing `identity:get_user(user_id=9e725435b0294296a28451de9a0d08c4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:12.885770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c474072-39df-4349-989f-bbc8cc018162 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:12.887500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 39/76] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:12 2022] GET /identity/v3/users/9e725435b0294296a28451de9a0d08c4 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:12.888266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c474072-39df-4349-989f-bbc8cc018162 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:12.926708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:12.962028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:12.973421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 38/77] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:12.974849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a6fbc-0cce-4887-958c-1964c6d1e8dd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:12.975138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a6fbc-0cce-4887-958c-1964c6d1e8dd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:12.975413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a6fbc-0cce-4887-958c-1964c6d1e8dd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:12.977814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 39/78] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:12.977814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a6fbc-0cce-4887-958c-1964c6d1e8dd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.003226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c474072-39df-4349-989f-bbc8cc018162 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.035400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd64c3-420b-4288-9766-eac1691ff036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.036025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd64c3-420b-4288-9766-eac1691ff036 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.036310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd64c3-420b-4288-9766-eac1691ff036 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.036585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd64c3-420b-4288-9766-eac1691ff036 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.037303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd64c3-420b-4288-9766-eac1691ff036 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:13.037607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd64c3-420b-4288-9766-eac1691ff036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:13.039027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dd64c3-420b-4288-9766-eac1691ff036 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:13.046957 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 40/79] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:13.047739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd64c3-420b-4288-9766-eac1691ff036 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.076609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a6fbc-0cce-4887-958c-1964c6d1e8dd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.110614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263747c-5ef8-4fe5-a1f1-4492f362e41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.111269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263747c-5ef8-4fe5-a1f1-4492f362e41e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.111507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263747c-5ef8-4fe5-a1f1-4492f362e41e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.111871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263747c-5ef8-4fe5-a1f1-4492f362e41e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.122973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263747c-5ef8-4fe5-a1f1-4492f362e41e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9263747c-5ef8-4fe5-a1f1-4492f362e41e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:13.128016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:13.128895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:13.129628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 40/80] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:13.130703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263747c-5ef8-4fe5-a1f1-4492f362e41e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.148860 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd64c3-420b-4288-9766-eac1691ff036 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.151640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:13.176351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:13.182681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650bd4d-bdee-487f-8abf-d1d187b34daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.183253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650bd4d-bdee-487f-8abf-d1d187b34daa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.183537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650bd4d-bdee-487f-8abf-d1d187b34daa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.183810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650bd4d-bdee-487f-8abf-d1d187b34daa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.230761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263747c-5ef8-4fe5-a1f1-4492f362e41e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.253228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650bd4d-bdee-487f-8abf-d1d187b34daa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:13.254740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650bd4d-bdee-487f-8abf-d1d187b34daa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:13.265419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99de842-c098-45f3-a398-fb95e6b4e34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.266099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99de842-c098-45f3-a398-fb95e6b4e34a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.266380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99de842-c098-45f3-a398-fb95e6b4e34a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.266652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99de842-c098-45f3-a398-fb95e6b4e34a None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.267853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99de842-c098-45f3-a398-fb95e6b4e34a None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:13.269337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99de842-c098-45f3-a398-fb95e6b4e34a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:13.271087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 41/81] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:13 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:13.272200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8d20-0bc2-4aec-865b-490e76eb1efd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.272486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8d20-0bc2-4aec-865b-490e76eb1efd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.272762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097d8d20-0bc2-4aec-865b-490e76eb1efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.290364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-097d8d20-0bc2-4aec-865b-490e76eb1efd None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:13.291542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-097d8d20-0bc2-4aec-865b-490e76eb1efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:13.356394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 41/82] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:13.357119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 42/83] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:13.357412 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650bd4d-bdee-487f-8abf-d1d187b34daa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.361864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d8d20-0bc2-4aec-865b-490e76eb1efd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.379930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:13.388062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:13.459838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650bd4d-bdee-487f-8abf-d1d187b34daa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.461163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097d8d20-0bc2-4aec-865b-490e76eb1efd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.493750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a6a1d-7ae0-4f6d-a9eb-faadd51ed493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.494427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a6a1d-7ae0-4f6d-a9eb-faadd51ed493 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.494710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a6a1d-7ae0-4f6d-a9eb-faadd51ed493 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.494908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20ff8e-93ac-4565-89fe-7838848fb6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.495015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0a6a1d-7ae0-4f6d-a9eb-faadd51ed493 None admin] PATH_INFO: `/v3/users/5c36c5fbc8c04f3f8235a61fc5992850` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.495525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20ff8e-93ac-4565-89fe-7838848fb6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.495793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20ff8e-93ac-4565-89fe-7838848fb6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.496066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e20ff8e-93ac-4565-89fe-7838848fb6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.496153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a6a1d-7ae0-4f6d-a9eb-faadd51ed493 None admin] RBAC: Authorizing `identity:get_user(user_id=5c36c5fbc8c04f3f8235a61fc5992850)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:13.496783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20ff8e-93ac-4565-89fe-7838848fb6e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:13.497087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20ff8e-93ac-4565-89fe-7838848fb6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:13.497580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0a6a1d-7ae0-4f6d-a9eb-faadd51ed493 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:13.498491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e20ff8e-93ac-4565-89fe-7838848fb6e9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:13.499270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 42/84] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:13 2022] GET /identity/v3/users/5c36c5fbc8c04f3f8235a61fc5992850 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:13.500030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a6a1d-7ae0-4f6d-a9eb-faadd51ed493 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.506997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 43/85] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:13.507902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20ff8e-93ac-4565-89fe-7838848fb6e9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.588462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:13.599919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:13.604200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0a6a1d-7ae0-4f6d-a9eb-faadd51ed493 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.610167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e20ff8e-93ac-4565-89fe-7838848fb6e9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.638378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e66301-863b-447e-a990-786f60d4b257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.638998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e66301-863b-447e-a990-786f60d4b257 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.639279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e66301-863b-447e-a990-786f60d4b257 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.639550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e66301-863b-447e-a990-786f60d4b257 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.640335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e66301-863b-447e-a990-786f60d4b257 None admin] RBAC: Adding query filter params (role.id=64dfc66a8abe4468ac4f1ca071d28bd4, scope.project.id=9ad9190f63064d308a0423e34054b7df, user.id=5c36c5fbc8c04f3f8235a61fc5992850) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:13.640644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e66301-863b-447e-a990-786f60d4b257 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:13.642130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e66301-863b-447e-a990-786f60d4b257 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:13.642606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a2792-e3bf-41d3-830f-0c8364ec9ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.643303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a2792-e3bf-41d3-830f-0c8364ec9ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.643593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a2792-e3bf-41d3-830f-0c8364ec9ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.643874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a2792-e3bf-41d3-830f-0c8364ec9ff8 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.645041 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a2792-e3bf-41d3-830f-0c8364ec9ff8 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:13.646506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6a2792-e3bf-41d3-830f-0c8364ec9ff8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:13.647459 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 43/86] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:13 2022] GET /identity/v3/role_assignments?user.id=5c36c5fbc8c04f3f8235a61fc5992850&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:13.653175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 44/87] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:13 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:13.653345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e66301-863b-447e-a990-786f60d4b257 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.653418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a2792-e3bf-41d3-830f-0c8364ec9ff8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.770172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e66301-863b-447e-a990-786f60d4b257 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.770735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6a2792-e3bf-41d3-830f-0c8364ec9ff8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.808820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8ae51-1abf-4c03-9f89-c04c1b528b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.809139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8ae51-1abf-4c03-9f89-c04c1b528b32 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.809398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8ae51-1abf-4c03-9f89-c04c1b528b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.809639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8ae51-1abf-4c03-9f89-c04c1b528b32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.810100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a678-fd51-41a9-b7a6-be549c8accfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.811459 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a678-fd51-41a9-b7a6-be549c8accfe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.811459 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a678-fd51-41a9-b7a6-be549c8accfe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.811459 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d0a678-fd51-41a9-b7a6-be549c8accfe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.813415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a678-fd51-41a9-b7a6-be549c8accfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:13.814937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d0a678-fd51-41a9-b7a6-be549c8accfe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:13.816840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 44/88] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:13.818661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a678-fd51-41a9-b7a6-be549c8accfe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.830823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:13.853672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:13.882707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8ae51-1abf-4c03-9f89-c04c1b528b32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:13.884269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef8ae51-1abf-4c03-9f89-c04c1b528b32 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:13.927937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d0a678-fd51-41a9-b7a6-be549c8accfe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:13.962482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae8a33-b009-4b0f-8083-24338339003a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:13.963139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae8a33-b009-4b0f-8083-24338339003a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:13.963424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae8a33-b009-4b0f-8083-24338339003a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:13.963698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae8a33-b009-4b0f-8083-24338339003a None admin] PATH_INFO: `/v3/users/5c36c5fbc8c04f3f8235a61fc5992850` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:13.964869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae8a33-b009-4b0f-8083-24338339003a None admin] RBAC: Authorizing `identity:get_user(user_id=5c36c5fbc8c04f3f8235a61fc5992850)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:13.966762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daae8a33-b009-4b0f-8083-24338339003a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:13.968959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 45/89] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:13 2022] GET /identity/v3/users/5c36c5fbc8c04f3f8235a61fc5992850 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:13.974087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae8a33-b009-4b0f-8083-24338339003a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:13.997729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 45/90] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:13.998904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8ae51-1abf-4c03-9f89-c04c1b528b32 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:14.049135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:14.090413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:14.094398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae8a33-b009-4b0f-8083-24338339003a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:14.102767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8ae51-1abf-4c03-9f89-c04c1b528b32 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:14.137343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca837b1-41c6-4551-bb9e-aeabd6650817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:14.137528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee66da-1c96-4501-b09a-4a21be2f9e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:14.138005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca837b1-41c6-4551-bb9e-aeabd6650817 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:14.138132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee66da-1c96-4501-b09a-4a21be2f9e0a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:14.138285 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca837b1-41c6-4551-bb9e-aeabd6650817 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:14.138374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee66da-1c96-4501-b09a-4a21be2f9e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:14.138559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca837b1-41c6-4551-bb9e-aeabd6650817 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:14.138649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ee66da-1c96-4501-b09a-4a21be2f9e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:14.139318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca837b1-41c6-4551-bb9e-aeabd6650817 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:14.139408 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee66da-1c96-4501-b09a-4a21be2f9e0a None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=9ad9190f63064d308a0423e34054b7df, user.id=9e725435b0294296a28451de9a0d08c4) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:14.139655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee66da-1c96-4501-b09a-4a21be2f9e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:14.140829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca837b1-41c6-4551-bb9e-aeabd6650817 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:14.141090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ee66da-1c96-4501-b09a-4a21be2f9e0a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:14.153335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 46/91] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:13 2022] GET /identity/v3/role_assignments?user.id=9e725435b0294296a28451de9a0d08c4&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:14.153335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee66da-1c96-4501-b09a-4a21be2f9e0a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:14.165905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 46/92] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:11:13 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:11:14.166875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca837b1-41c6-4551-bb9e-aeabd6650817 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:14.280232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca837b1-41c6-4551-bb9e-aeabd6650817 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:14.283268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ee66da-1c96-4501-b09a-4a21be2f9e0a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:14.283838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:14.320535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64d9f4-bd23-487c-9d44-c926c3c1f607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:14.321136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64d9f4-bd23-487c-9d44-c926c3c1f607 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:14.321431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64d9f4-bd23-487c-9d44-c926c3c1f607 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:14.321719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64d9f4-bd23-487c-9d44-c926c3c1f607 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:14.323087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64d9f4-bd23-487c-9d44-c926c3c1f607 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:14.324644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64d9f4-bd23-487c-9d44-c926c3c1f607 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:14.326535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 47/93] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:14 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:14.333061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06af16e-561a-4cc6-94cc-8b3059956453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:14.333296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64d9f4-bd23-487c-9d44-c926c3c1f607 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:14.333842 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06af16e-561a-4cc6-94cc-8b3059956453 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:14.334135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06af16e-561a-4cc6-94cc-8b3059956453 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:14.334456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06af16e-561a-4cc6-94cc-8b3059956453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:14.385498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:14.438015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06af16e-561a-4cc6-94cc-8b3059956453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:14.440603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06af16e-561a-4cc6-94cc-8b3059956453 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:14.487089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64d9f4-bd23-487c-9d44-c926c3c1f607 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:14.527047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cdc14-27dc-40bc-9d86-12682b1ed6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:14.527687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cdc14-27dc-40bc-9d86-12682b1ed6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:14.527971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cdc14-27dc-40bc-9d86-12682b1ed6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:14.528248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99cdc14-27dc-40bc-9d86-12682b1ed6f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:14.529159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cdc14-27dc-40bc-9d86-12682b1ed6f3 None admin] RBAC: Adding query filter params (role.id=5e0c08646046426385776839f1f168cd, scope.project.id=9ad9190f63064d308a0423e34054b7df, user.id=5c36c5fbc8c04f3f8235a61fc5992850) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:14.529476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cdc14-27dc-40bc-9d86-12682b1ed6f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:14.531110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99cdc14-27dc-40bc-9d86-12682b1ed6f3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:14.542816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 48/94] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:14 2022] GET /identity/v3/role_assignments?user.id=5c36c5fbc8c04f3f8235a61fc5992850&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=5e0c08646046426385776839f1f168cd => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:14.572540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 47/95] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:14.580516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cdc14-27dc-40bc-9d86-12682b1ed6f3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:14.629825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:14.703466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99cdc14-27dc-40bc-9d86-12682b1ed6f3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:14.743098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a121c1-4efa-4ecc-a1e2-07d0ad1caf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:14.743764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a121c1-4efa-4ecc-a1e2-07d0ad1caf21 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:14.744052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a121c1-4efa-4ecc-a1e2-07d0ad1caf21 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:14.744334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a121c1-4efa-4ecc-a1e2-07d0ad1caf21 None admin] PATH_INFO: `/v3/projects/9af2f393366b4f8b8299ac7d763b021a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:14.745530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a121c1-4efa-4ecc-a1e2-07d0ad1caf21 None admin] RBAC: Authorizing `identity:get_project(project_id=9af2f393366b4f8b8299ac7d763b021a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:14.747081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a121c1-4efa-4ecc-a1e2-07d0ad1caf21 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:14.748995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 49/96] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:14 2022] GET /identity/v3/projects/9af2f393366b4f8b8299ac7d763b021a => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:14.754883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06af16e-561a-4cc6-94cc-8b3059956453 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:14.825809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:14.871327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06af16e-561a-4cc6-94cc-8b3059956453 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:14.910404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b26b97-d4b0-4b8c-9a0b-257102a193f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:14.911474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b26b97-d4b0-4b8c-9a0b-257102a193f6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:14.911998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b26b97-d4b0-4b8c-9a0b-257102a193f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:14.912464 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b26b97-d4b0-4b8c-9a0b-257102a193f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:14.913457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b26b97-d4b0-4b8c-9a0b-257102a193f6 None admin] RBAC: Adding query filter params (role.id=64dfc66a8abe4468ac4f1ca071d28bd4, scope.project.id=9af2f393366b4f8b8299ac7d763b021a, user.id=5c36c5fbc8c04f3f8235a61fc5992850) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:14.914019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b26b97-d4b0-4b8c-9a0b-257102a193f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:14.915685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b26b97-d4b0-4b8c-9a0b-257102a193f6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:14.923130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 48/97] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:14 2022] GET /identity/v3/role_assignments?user.id=5c36c5fbc8c04f3f8235a61fc5992850&scope.project.id=9af2f393366b4f8b8299ac7d763b021a&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:15.977353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1e1ba-2cd8-40d4-b8c4-d6e9723fec96 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:15.977929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1e1ba-2cd8-40d4-b8c4-d6e9723fec96 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:15.977929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1e1ba-2cd8-40d4-b8c4-d6e9723fec96 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:15.979442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 50/98] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:15.985218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5b7e-2d99-4f8e-aac2-ab955e1de94e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:15.985606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5b7e-2d99-4f8e-aac2-ab955e1de94e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:15.985788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce5b7e-2d99-4f8e-aac2-ab955e1de94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:16.010050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-71ce5b7e-2d99-4f8e-aac2-ab955e1de94e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:16.011567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-71ce5b7e-2d99-4f8e-aac2-ab955e1de94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:16.096993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 49/99] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:16.140762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1e1ba-2cd8-40d4-b8c4-d6e9723fec96 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:16.270398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1e1ba-2cd8-40d4-b8c4-d6e9723fec96 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:16.272728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:16.312722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acfa73-979c-40d8-aee0-e13a2f84d9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:16.313377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acfa73-979c-40d8-aee0-e13a2f84d9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:16.313656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acfa73-979c-40d8-aee0-e13a2f84d9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:16.313954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33acfa73-979c-40d8-aee0-e13a2f84d9c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:16.326607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33acfa73-979c-40d8-aee0-e13a2f84d9c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-33acfa73-979c-40d8-aee0-e13a2f84d9c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:16.331695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:16.332543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:16.333290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 51/100] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:16.340385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce5b7e-2d99-4f8e-aac2-ab955e1de94e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:16.408070 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:16.464628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce5b7e-2d99-4f8e-aac2-ab955e1de94e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:16.509430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc2868-0a12-4d8e-b345-028c4490c85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:16.510101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc2868-0a12-4d8e-b345-028c4490c85c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:16.510380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc2868-0a12-4d8e-b345-028c4490c85c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:16.510650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dc2868-0a12-4d8e-b345-028c4490c85c None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:16.511376 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc2868-0a12-4d8e-b345-028c4490c85c None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:16.511682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc2868-0a12-4d8e-b345-028c4490c85c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:16.513060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dc2868-0a12-4d8e-b345-028c4490c85c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:16.525067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 50/101] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:16.531025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acfa73-979c-40d8-aee0-e13a2f84d9c1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:16.633081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33acfa73-979c-40d8-aee0-e13a2f84d9c1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:16.673787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78293787-0bb8-4ca7-9dd9-a4c9c92b8493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:16.674481 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78293787-0bb8-4ca7-9dd9-a4c9c92b8493 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:16.674846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78293787-0bb8-4ca7-9dd9-a4c9c92b8493 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:16.675121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78293787-0bb8-4ca7-9dd9-a4c9c92b8493 None admin] PATH_INFO: `/v3/users/5c36c5fbc8c04f3f8235a61fc5992850` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:16.682956 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78293787-0bb8-4ca7-9dd9-a4c9c92b8493 None admin] RBAC: Authorizing `identity:get_user(user_id=5c36c5fbc8c04f3f8235a61fc5992850)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:16.684462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78293787-0bb8-4ca7-9dd9-a4c9c92b8493 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:16.686233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 52/102] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:16 2022] GET /identity/v3/users/5c36c5fbc8c04f3f8235a61fc5992850 => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:16.691972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc2868-0a12-4d8e-b345-028c4490c85c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:16.817296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dc2868-0a12-4d8e-b345-028c4490c85c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:16.847481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:16.868602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a97dc2-28e8-4221-a75b-5a14bd088772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:16.869279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a97dc2-28e8-4221-a75b-5a14bd088772 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:16.869607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a97dc2-28e8-4221-a75b-5a14bd088772 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:16.869912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a97dc2-28e8-4221-a75b-5a14bd088772 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:16.874475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a97dc2-28e8-4221-a75b-5a14bd088772 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:16.875956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a97dc2-28e8-4221-a75b-5a14bd088772 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:16.877774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 51/103] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:16 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:16.888380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78293787-0bb8-4ca7-9dd9-a4c9c92b8493 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:16.927394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:17.004918 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78293787-0bb8-4ca7-9dd9-a4c9c92b8493 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:17.037825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc521a52-9b88-4bf9-a528-27b20be1a10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:17.038511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc521a52-9b88-4bf9-a528-27b20be1a10b None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:17.038795 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc521a52-9b88-4bf9-a528-27b20be1a10b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:17.039075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc521a52-9b88-4bf9-a528-27b20be1a10b None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df/users/5c36c5fbc8c04f3f8235a61fc5992850/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:17.041038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc521a52-9b88-4bf9-a528-27b20be1a10b None admin] RBAC: Authorizing `identity:create_grant(project_id=9ad9190f63064d308a0423e34054b7df, user_id=5c36c5fbc8c04f3f8235a61fc5992850, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:17.042602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc521a52-9b88-4bf9-a528-27b20be1a10b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:17.067485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 53/104] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:11:16 2022] PUT /identity/v3/projects/9ad9190f63064d308a0423e34054b7df/users/5c36c5fbc8c04f3f8235a61fc5992850/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:17.203399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326ad45-21ec-4aca-b965-e3fc2619f970 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:17.203399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326ad45-21ec-4aca-b965-e3fc2619f970 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:17.203399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326ad45-21ec-4aca-b965-e3fc2619f970 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:17.203399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 52/105] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:17.209622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162806e-949c-4763-a586-2f92f9d84aa9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:17.210026 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162806e-949c-4763-a586-2f92f9d84aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:17.210207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f162806e-949c-4763-a586-2f92f9d84aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:17.246904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f162806e-949c-4763-a586-2f92f9d84aa9 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:17.248134 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f162806e-949c-4763-a586-2f92f9d84aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:17.343413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 54/106] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:17.377518 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326ad45-21ec-4aca-b965-e3fc2619f970 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:17.492539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11f86f-3af9-429d-9f2c-4229e6d138ca None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:17.493056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11f86f-3af9-429d-9f2c-4229e6d138ca None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:17.493467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f11f86f-3af9-429d-9f2c-4229e6d138ca None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:17.495025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 55/107] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:11:17.500145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06c0c0-439b-48c9-a0c1-d1fd88870376 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:17.500590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06c0c0-439b-48c9-a0c1-d1fd88870376 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:17.501112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06c0c0-439b-48c9-a0c1-d1fd88870376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:17.514230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326ad45-21ec-4aca-b965-e3fc2619f970 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:17.536435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6e06c0c0-439b-48c9-a0c1-d1fd88870376 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:17.537930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6e06c0c0-439b-48c9-a0c1-d1fd88870376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:17.584163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:17.589547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273e222-715f-41e1-ae89-bd23ccc6719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:17.590319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273e222-715f-41e1-ae89-bd23ccc6719c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:17.590735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273e222-715f-41e1-ae89-bd23ccc6719c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:17.591068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273e222-715f-41e1-ae89-bd23ccc6719c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:17.616059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273e222-715f-41e1-ae89-bd23ccc6719c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c273e222-715f-41e1-ae89-bd23ccc6719c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:17.634591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:17.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:17.636221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 53/108] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:11:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:17.641575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273e222-715f-41e1-ae89-bd23ccc6719c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:17.693162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 56/109] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:17.722194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:17.726652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06c0c0-439b-48c9-a0c1-d1fd88870376 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:17.825618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273e222-715f-41e1-ae89-bd23ccc6719c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:17.883826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06c0c0-439b-48c9-a0c1-d1fd88870376 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:17.920495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85471856-1faf-4406-8309-b3542aab127b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:17.921517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85471856-1faf-4406-8309-b3542aab127b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:17.922019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85471856-1faf-4406-8309-b3542aab127b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:17.922481 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85471856-1faf-4406-8309-b3542aab127b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:17.933697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85471856-1faf-4406-8309-b3542aab127b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-85471856-1faf-4406-8309-b3542aab127b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:17.940711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:17.941529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:17.942307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:17.942856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f879c11a-b81e-48ae-8cc3-311db6ec62cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:17.943429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879c11a-b81e-48ae-8cc3-311db6ec62cd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:17.943715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879c11a-b81e-48ae-8cc3-311db6ec62cd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:17.944000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f879c11a-b81e-48ae-8cc3-311db6ec62cd None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:17.944735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879c11a-b81e-48ae-8cc3-311db6ec62cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:17.945041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879c11a-b81e-48ae-8cc3-311db6ec62cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:17.945685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 57/110] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:17.946478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f879c11a-b81e-48ae-8cc3-311db6ec62cd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:17.948026 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944aeeb7-6a94-48e6-8bb1-ee63d28af40d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:17.948502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944aeeb7-6a94-48e6-8bb1-ee63d28af40d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:17.949016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944aeeb7-6a94-48e6-8bb1-ee63d28af40d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:17.950485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 58/111] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:17.956755 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944aeeb7-6a94-48e6-8bb1-ee63d28af40d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:17.964430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 54/112] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:17.965727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f384ce8-2834-4308-a16c-45b1d516ca36 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:17.966046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f384ce8-2834-4308-a16c-45b1d516ca36 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:17.966330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f384ce8-2834-4308-a16c-45b1d516ca36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:17.987743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:17.990370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5f384ce8-2834-4308-a16c-45b1d516ca36 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:17.991669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f384ce8-2834-4308-a16c-45b1d516ca36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:18.060467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 55/113] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:18.069933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51cace-aaa8-49ea-8291-72ebe0aa0cd5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.069933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51cace-aaa8-49ea-8291-72ebe0aa0cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.069933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51cace-aaa8-49ea-8291-72ebe0aa0cd5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.069933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 56/114] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:18.070870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51cace-aaa8-49ea-8291-72ebe0aa0cd5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:18.084929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944aeeb7-6a94-48e6-8bb1-ee63d28af40d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:18.121073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a4f8f-655b-44a6-a7b5-bf0a380942e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:18.121715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a4f8f-655b-44a6-a7b5-bf0a380942e2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.122031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a4f8f-655b-44a6-a7b5-bf0a380942e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.122313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a4f8f-655b-44a6-a7b5-bf0a380942e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.123045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a4f8f-655b-44a6-a7b5-bf0a380942e2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:18.123150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:18.123358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a4f8f-655b-44a6-a7b5-bf0a380942e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:18.124756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a4f8f-655b-44a6-a7b5-bf0a380942e2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:18.133357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 59/115] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:18.134493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f8cc-63b2-49aa-9de3-fed03aa592d4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.134732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f8cc-63b2-49aa-9de3-fed03aa592d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.135013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f08f8cc-63b2-49aa-9de3-fed03aa592d4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.140162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 60/116] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:18.140162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec930346-5a00-4bce-86d2-9a300fc3fb8e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.140162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec930346-5a00-4bce-86d2-9a300fc3fb8e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.140162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec930346-5a00-4bce-86d2-9a300fc3fb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.157646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ec930346-5a00-4bce-86d2-9a300fc3fb8e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:18.159014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec930346-5a00-4bce-86d2-9a300fc3fb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:18.200381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51cace-aaa8-49ea-8291-72ebe0aa0cd5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:18.237231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 61/117] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:18.237231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec930346-5a00-4bce-86d2-9a300fc3fb8e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:18.238758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:18.247838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43a13-d5b0-4cef-9566-3eb38329dde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:18.248508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43a13-d5b0-4cef-9566-3eb38329dde4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.248787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43a13-d5b0-4cef-9566-3eb38329dde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.249058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa43a13-d5b0-4cef-9566-3eb38329dde4 None admin] PATH_INFO: `/v3/users/9e725435b0294296a28451de9a0d08c4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.251379 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa43a13-d5b0-4cef-9566-3eb38329dde4 None admin] RBAC: Authorizing `identity:get_user(user_id=9e725435b0294296a28451de9a0d08c4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:18.253027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa43a13-d5b0-4cef-9566-3eb38329dde4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:18.255481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 57/118] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:18 2022] GET /identity/v3/users/9e725435b0294296a28451de9a0d08c4 => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:18.260237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43a13-d5b0-4cef-9566-3eb38329dde4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:18.352568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa43a13-d5b0-4cef-9566-3eb38329dde4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:18.360929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec930346-5a00-4bce-86d2-9a300fc3fb8e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:18.361084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:18.390017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b689c8e9-02f5-4928-82f6-74d56a798f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:18.390726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689c8e9-02f5-4928-82f6-74d56a798f53 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.391009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689c8e9-02f5-4928-82f6-74d56a798f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.391282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b689c8e9-02f5-4928-82f6-74d56a798f53 None admin] PATH_INFO: `/v3/users/5c36c5fbc8c04f3f8235a61fc5992850` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.392629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689c8e9-02f5-4928-82f6-74d56a798f53 None admin] RBAC: Authorizing `identity:get_user(user_id=5c36c5fbc8c04f3f8235a61fc5992850)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:18.393995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b689c8e9-02f5-4928-82f6-74d56a798f53 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:18.395759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 58/119] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:18 2022] GET /identity/v3/users/5c36c5fbc8c04f3f8235a61fc5992850 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:18.398297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b888af-6ea5-4ff1-ab80-bf471a182eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:18.398764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b888af-6ea5-4ff1-ab80-bf471a182eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.398764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b888af-6ea5-4ff1-ab80-bf471a182eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.398764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b888af-6ea5-4ff1-ab80-bf471a182eb8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.401295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629718d-cd2a-42c3-bff2-3cb3b7b84b34 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.401591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629718d-cd2a-42c3-bff2-3cb3b7b84b34 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.401867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629718d-cd2a-42c3-bff2-3cb3b7b84b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.412949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b888af-6ea5-4ff1-ab80-bf471a182eb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-70b888af-6ea5-4ff1-ab80-bf471a182eb8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:18.417961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:18.418775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:18.419528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 62/120] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:18.420607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b888af-6ea5-4ff1-ab80-bf471a182eb8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:18.425935 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e629718d-cd2a-42c3-bff2-3cb3b7b84b34 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:18.426447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e629718d-cd2a-42c3-bff2-3cb3b7b84b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:18.489571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:18.502151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 59/121] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:18.503203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629718d-cd2a-42c3-bff2-3cb3b7b84b34 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:18.536552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b888af-6ea5-4ff1-ab80-bf471a182eb8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:18.571005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f04ec-ef8e-4909-8504-4311118b69a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:18.571758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f04ec-ef8e-4909-8504-4311118b69a0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.572045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f04ec-ef8e-4909-8504-4311118b69a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.572325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f04ec-ef8e-4909-8504-4311118b69a0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.583323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f04ec-ef8e-4909-8504-4311118b69a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-767f04ec-ef8e-4909-8504-4311118b69a0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:18.588742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:18.589603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:18.590631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 63/122] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:18.595465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f04ec-ef8e-4909-8504-4311118b69a0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:18.618072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629718d-cd2a-42c3-bff2-3cb3b7b84b34 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:18.626622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:18.656823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e2cba-29a0-4d96-b6d8-5d92b31f19e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:18.657557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e2cba-29a0-4d96-b6d8-5d92b31f19e4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.657872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e2cba-29a0-4d96-b6d8-5d92b31f19e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.658151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e2cba-29a0-4d96-b6d8-5d92b31f19e4 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.659482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e2cba-29a0-4d96-b6d8-5d92b31f19e4 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:18.661207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274e2cba-29a0-4d96-b6d8-5d92b31f19e4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:18.663212 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 60/123] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:18 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:18.663912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e2cba-29a0-4d96-b6d8-5d92b31f19e4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:18.707323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f04ec-ef8e-4909-8504-4311118b69a0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:18.732131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:18.744244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa7e85-2707-4e98-91ad-d474a5eee784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:18.745020 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa7e85-2707-4e98-91ad-d474a5eee784 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.745310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa7e85-2707-4e98-91ad-d474a5eee784 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.745706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fa7e85-2707-4e98-91ad-d474a5eee784 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.746903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa7e85-2707-4e98-91ad-d474a5eee784 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:18.748503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fa7e85-2707-4e98-91ad-d474a5eee784 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:18.756179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 64/124] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:18 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:18.756179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa7e85-2707-4e98-91ad-d474a5eee784 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:18.770123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274e2cba-29a0-4d96-b6d8-5d92b31f19e4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:18.802871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd315014-213d-495a-9b2b-5406526d4f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:18.803628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd315014-213d-495a-9b2b-5406526d4f02 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.803975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd315014-213d-495a-9b2b-5406526d4f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.804311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd315014-213d-495a-9b2b-5406526d4f02 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.805095 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd315014-213d-495a-9b2b-5406526d4f02 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:18.805465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd315014-213d-495a-9b2b-5406526d4f02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:18.806949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd315014-213d-495a-9b2b-5406526d4f02 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:18.815076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 61/125] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:18.820417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd315014-213d-495a-9b2b-5406526d4f02 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:18.856783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:18.861181 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fa7e85-2707-4e98-91ad-d474a5eee784 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:18.895552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fdec7-9d70-4fac-a47a-5a87934f1c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:18.896298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fdec7-9d70-4fac-a47a-5a87934f1c5e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.896653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fdec7-9d70-4fac-a47a-5a87934f1c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.896928 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fdec7-9d70-4fac-a47a-5a87934f1c5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.898115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fdec7-9d70-4fac-a47a-5a87934f1c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:18.899726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fdec7-9d70-4fac-a47a-5a87934f1c5e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:18.901341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 65/126] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:18.905794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fdec7-9d70-4fac-a47a-5a87934f1c5e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:18.949510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd315014-213d-495a-9b2b-5406526d4f02 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:18.971211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:18.986808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a124af7-3a33-4036-983d-21b5660e29c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:18.987520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a124af7-3a33-4036-983d-21b5660e29c1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:18.987799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a124af7-3a33-4036-983d-21b5660e29c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:18.988077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a124af7-3a33-4036-983d-21b5660e29c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:18.988819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a124af7-3a33-4036-983d-21b5660e29c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:18.989122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a124af7-3a33-4036-983d-21b5660e29c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:18.990564 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a124af7-3a33-4036-983d-21b5660e29c1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:18.999368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 62/127] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:19.000043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a124af7-3a33-4036-983d-21b5660e29c1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:19.020595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fdec7-9d70-4fac-a47a-5a87934f1c5e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:19.057640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3a37f-25a9-43d9-b0b2-173ef039b002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:19.058370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3a37f-25a9-43d9-b0b2-173ef039b002 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:19.058655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3a37f-25a9-43d9-b0b2-173ef039b002 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:19.058931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3a37f-25a9-43d9-b0b2-173ef039b002 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df/users/9e725435b0294296a28451de9a0d08c4/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:19.060966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3a37f-25a9-43d9-b0b2-173ef039b002 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ad9190f63064d308a0423e34054b7df, user_id=9e725435b0294296a28451de9a0d08c4, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:19.062446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3a37f-25a9-43d9-b0b2-173ef039b002 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:19.085783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 66/128] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:11:18 2022] PUT /identity/v3/projects/9ad9190f63064d308a0423e34054b7df/users/9e725435b0294296a28451de9a0d08c4/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:19.085783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3a37f-25a9-43d9-b0b2-173ef039b002 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:19.107762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:19.115945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a124af7-3a33-4036-983d-21b5660e29c1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:19.163333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccdd9c-6ed6-45ab-b144-22d41da6f87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:19.164187 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccdd9c-6ed6-45ab-b144-22d41da6f87e None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:19.164634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccdd9c-6ed6-45ab-b144-22d41da6f87e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:19.164974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ccdd9c-6ed6-45ab-b144-22d41da6f87e None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df/users/5c36c5fbc8c04f3f8235a61fc5992850/roles/5e0c08646046426385776839f1f168cd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:19.167666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccdd9c-6ed6-45ab-b144-22d41da6f87e None admin] RBAC: Authorizing `identity:create_grant(project_id=9ad9190f63064d308a0423e34054b7df, user_id=5c36c5fbc8c04f3f8235a61fc5992850, role_id=5e0c08646046426385776839f1f168cd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:19.169539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ccdd9c-6ed6-45ab-b144-22d41da6f87e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:19.189486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 63/129] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:11:18 2022] PUT /identity/v3/projects/9ad9190f63064d308a0423e34054b7df/users/5c36c5fbc8c04f3f8235a61fc5992850/roles/5e0c08646046426385776839f1f168cd => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:19.192557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccdd9c-6ed6-45ab-b144-22d41da6f87e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:19.257189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3a37f-25a9-43d9-b0b2-173ef039b002 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:19.289786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:19.313782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c3a2f-5d87-47f9-8694-df74dc06504e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:19.314541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c3a2f-5d87-47f9-8694-df74dc06504e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:19.315174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c3a2f-5d87-47f9-8694-df74dc06504e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:19.315491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270c3a2f-5d87-47f9-8694-df74dc06504e None admin] PATH_INFO: `/v3/users/55d6f7dcb65a491c9baea3c593780f8a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:19.339602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c3a2f-5d87-47f9-8694-df74dc06504e None admin] RBAC: Authorizing `identity:get_user(user_id=55d6f7dcb65a491c9baea3c593780f8a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:19.341096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270c3a2f-5d87-47f9-8694-df74dc06504e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:19.343523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 67/130] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:19 2022] GET /identity/v3/users/55d6f7dcb65a491c9baea3c593780f8a => generated 289 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:19.344341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c3a2f-5d87-47f9-8694-df74dc06504e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:19.359110 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ccdd9c-6ed6-45ab-b144-22d41da6f87e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:19.401195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed23b9-2fb7-4333-a39f-13d793025559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:19.401912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed23b9-2fb7-4333-a39f-13d793025559 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:19.402198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed23b9-2fb7-4333-a39f-13d793025559 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:19.402479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ed23b9-2fb7-4333-a39f-13d793025559 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:19.403189 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed23b9-2fb7-4333-a39f-13d793025559 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:19.404705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ed23b9-2fb7-4333-a39f-13d793025559 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:19.443508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 64/131] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:11:19 2022] POST /identity/v3/projects => generated 302 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:11:19.444415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed23b9-2fb7-4333-a39f-13d793025559 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:19.456863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:19.489264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270c3a2f-5d87-47f9-8694-df74dc06504e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:19.537224 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0efda2-5d49-4a8f-a6e6-cb635d0f167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:19.538499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0efda2-5d49-4a8f-a6e6-cb635d0f167d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:19.538783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0efda2-5d49-4a8f-a6e6-cb635d0f167d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:19.539068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0efda2-5d49-4a8f-a6e6-cb635d0f167d None admin] PATH_INFO: `/v3/users/5c36c5fbc8c04f3f8235a61fc5992850` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:19.543588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0efda2-5d49-4a8f-a6e6-cb635d0f167d None admin] RBAC: Authorizing `identity:get_user(user_id=5c36c5fbc8c04f3f8235a61fc5992850)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:19.545201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0efda2-5d49-4a8f-a6e6-cb635d0f167d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:19.547404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 68/132] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:19 2022] GET /identity/v3/users/5c36c5fbc8c04f3f8235a61fc5992850 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:19.553166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0efda2-5d49-4a8f-a6e6-cb635d0f167d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:19.584834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ed23b9-2fb7-4333-a39f-13d793025559 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:19.603701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:19.629407 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820604c-1bc7-426c-be8b-8bca7daf2151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:19.630280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820604c-1bc7-426c-be8b-8bca7daf2151 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:19.630665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820604c-1bc7-426c-be8b-8bca7daf2151 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:19.631041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820604c-1bc7-426c-be8b-8bca7daf2151 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:19.664070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0efda2-5d49-4a8f-a6e6-cb635d0f167d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:19.701189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5b6df-e389-493d-a995-2f10d97c1108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:19.702225 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5b6df-e389-493d-a995-2f10d97c1108 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:19.702655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5b6df-e389-493d-a995-2f10d97c1108 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:19.703131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5b6df-e389-493d-a995-2f10d97c1108 None admin] PATH_INFO: `/v3/projects/9af2f393366b4f8b8299ac7d763b021a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:19.704647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5b6df-e389-493d-a995-2f10d97c1108 None admin] RBAC: Authorizing `identity:get_project(project_id=9af2f393366b4f8b8299ac7d763b021a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:19.706469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5b6df-e389-493d-a995-2f10d97c1108 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:19.708535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820604c-1bc7-426c-be8b-8bca7daf2151 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:19.709080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 69/133] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:19 2022] GET /identity/v3/projects/9af2f393366b4f8b8299ac7d763b021a => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:19.714030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820604c-1bc7-426c-be8b-8bca7daf2151 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:19.715053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5b6df-e389-493d-a995-2f10d97c1108 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:19.721457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:19.824455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5b6df-e389-493d-a995-2f10d97c1108 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:19.826523 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 65/134] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:19.828129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b717374-24bc-4f51-99a2-db3ec4837c8d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:19.828129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b717374-24bc-4f51-99a2-db3ec4837c8d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:19.828353 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b717374-24bc-4f51-99a2-db3ec4837c8d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:19.829869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 66/135] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:19.834683 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b717374-24bc-4f51-99a2-db3ec4837c8d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:19.841031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:19.863954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf38ab-1c76-47c0-aff5-c705822202d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:19.864842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf38ab-1c76-47c0-aff5-c705822202d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:19.864976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf38ab-1c76-47c0-aff5-c705822202d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:19.865194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf38ab-1c76-47c0-aff5-c705822202d6 None admin] PATH_INFO: `/v3/projects/9af2f393366b4f8b8299ac7d763b021a/users/5c36c5fbc8c04f3f8235a61fc5992850/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:19.867292 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf38ab-1c76-47c0-aff5-c705822202d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9af2f393366b4f8b8299ac7d763b021a, user_id=5c36c5fbc8c04f3f8235a61fc5992850, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:19.868770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf38ab-1c76-47c0-aff5-c705822202d6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:19.899049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 70/136] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:11:19 2022] PUT /identity/v3/projects/9af2f393366b4f8b8299ac7d763b021a/users/5c36c5fbc8c04f3f8235a61fc5992850/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:19.900924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23125437-fdcf-468f-aca2-cf083f77665f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:19.901394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23125437-fdcf-468f-aca2-cf083f77665f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:19.902199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23125437-fdcf-468f-aca2-cf083f77665f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:19.928489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-23125437-fdcf-468f-aca2-cf083f77665f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:19.930416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-23125437-fdcf-468f-aca2-cf083f77665f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:19.971989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b717374-24bc-4f51-99a2-db3ec4837c8d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:19.985900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:20.012863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 71/137] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:20.029600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbf918-5c2b-41f6-abc6-a6eca16a4f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:20.030546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbf918-5c2b-41f6-abc6-a6eca16a4f20 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:20.030963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbf918-5c2b-41f6-abc6-a6eca16a4f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:20.031369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbf918-5c2b-41f6-abc6-a6eca16a4f20 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:20.038652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbf918-5c2b-41f6-abc6-a6eca16a4f20 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:20.040385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dbf918-5c2b-41f6-abc6-a6eca16a4f20 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:20.042390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 67/138] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:19 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:20.048760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23125437-fdcf-468f-aca2-cf083f77665f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:20.052956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbf918-5c2b-41f6-abc6-a6eca16a4f20 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:20.174188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbf918-5c2b-41f6-abc6-a6eca16a4f20 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:20.219859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a3002-d906-4559-ae14-abd9bd0f2bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:20.221156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a3002-d906-4559-ae14-abd9bd0f2bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:20.221944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a3002-d906-4559-ae14-abd9bd0f2bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:20.222359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a3002-d906-4559-ae14-abd9bd0f2bc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:20.236812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a3002-d906-4559-ae14-abd9bd0f2bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0a4a3002-d906-4559-ae14-abd9bd0f2bc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:20.244628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:20.245665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:20.248318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:20.249031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23125437-fdcf-468f-aca2-cf083f77665f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:20.251297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 68/139] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:20.257788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:20.262988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a3002-d906-4559-ae14-abd9bd0f2bc4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:20.300022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468022d2-d063-4c6c-bac6-7cfef9000985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:20.301030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468022d2-d063-4c6c-bac6-7cfef9000985 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:20.301512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468022d2-d063-4c6c-bac6-7cfef9000985 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:20.302043 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468022d2-d063-4c6c-bac6-7cfef9000985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:20.303058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468022d2-d063-4c6c-bac6-7cfef9000985 None admin] RBAC: Adding query filter params (role.id=511279b616c245e2baad4d3a59cd477e, scope.project.id=9ad9190f63064d308a0423e34054b7df, user.id=55d6f7dcb65a491c9baea3c593780f8a) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:20.303566 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468022d2-d063-4c6c-bac6-7cfef9000985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:20.305222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468022d2-d063-4c6c-bac6-7cfef9000985 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:20.311238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 72/140] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:20 2022] GET /identity/v3/role_assignments?user.id=55d6f7dcb65a491c9baea3c593780f8a&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=511279b616c245e2baad4d3a59cd477e => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:20.339151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:20.450012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a3002-d906-4559-ae14-abd9bd0f2bc4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:20.508961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bdfa3-3180-4150-85db-9e69a8ac6952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:20.509619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bdfa3-3180-4150-85db-9e69a8ac6952 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:20.509931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bdfa3-3180-4150-85db-9e69a8ac6952 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:20.510211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bdfa3-3180-4150-85db-9e69a8ac6952 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:20.510955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bdfa3-3180-4150-85db-9e69a8ac6952 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:20.511273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bdfa3-3180-4150-85db-9e69a8ac6952 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:20.512687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bdfa3-3180-4150-85db-9e69a8ac6952 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:20.530503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 69/141] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:20.536757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468022d2-d063-4c6c-bac6-7cfef9000985 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:20.692514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468022d2-d063-4c6c-bac6-7cfef9000985 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:20.732352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75425a4-269f-45ad-8558-5d6f7569cd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:20.733784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75425a4-269f-45ad-8558-5d6f7569cd4b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:20.734293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75425a4-269f-45ad-8558-5d6f7569cd4b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:20.734763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75425a4-269f-45ad-8558-5d6f7569cd4b None admin] PATH_INFO: `/v3/users/5c36c5fbc8c04f3f8235a61fc5992850` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:20.736164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75425a4-269f-45ad-8558-5d6f7569cd4b None admin] RBAC: Authorizing `identity:get_user(user_id=5c36c5fbc8c04f3f8235a61fc5992850)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:20.737907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75425a4-269f-45ad-8558-5d6f7569cd4b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:20.739757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 73/142] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:20 2022] GET /identity/v3/users/5c36c5fbc8c04f3f8235a61fc5992850 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:20.746616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bdfa3-3180-4150-85db-9e69a8ac6952 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:20.918447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:20.922922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bdfa3-3180-4150-85db-9e69a8ac6952 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:20.969347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb02ca-b58e-4e33-b35e-4353b371eeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:20.969932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb02ca-b58e-4e33-b35e-4353b371eeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:20.970215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb02ca-b58e-4e33-b35e-4353b371eeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:20.970486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb02ca-b58e-4e33-b35e-4353b371eeb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:21.054872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb02ca-b58e-4e33-b35e-4353b371eeb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:21.056373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb02ca-b58e-4e33-b35e-4353b371eeb4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:21.178149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:21.185629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 70/143] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:21.194846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75425a4-269f-45ad-8558-5d6f7569cd4b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:21.251293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:21.329153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75425a4-269f-45ad-8558-5d6f7569cd4b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:21.369180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d113732-6067-408e-afad-7763e3b5a3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:21.370235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d113732-6067-408e-afad-7763e3b5a3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:21.370735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d113732-6067-408e-afad-7763e3b5a3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:21.371310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d113732-6067-408e-afad-7763e3b5a3b0 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:21.379407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d113732-6067-408e-afad-7763e3b5a3b0 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:21.381104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d113732-6067-408e-afad-7763e3b5a3b0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:21.383060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 74/144] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:21 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:21.394864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb02ca-b58e-4e33-b35e-4353b371eeb4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:21.499453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb02ca-b58e-4e33-b35e-4353b371eeb4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:21.537393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14798e06-8dc9-4e3f-821b-3b4c7d663e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:21.537980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14798e06-8dc9-4e3f-821b-3b4c7d663e87 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:21.538265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14798e06-8dc9-4e3f-821b-3b4c7d663e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:21.538540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14798e06-8dc9-4e3f-821b-3b4c7d663e87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:21.539340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14798e06-8dc9-4e3f-821b-3b4c7d663e87 None admin] RBAC: Adding query filter params (role.id=64dfc66a8abe4468ac4f1ca071d28bd4, scope.project.id=9ad9190f63064d308a0423e34054b7df, user.id=5c36c5fbc8c04f3f8235a61fc5992850) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:21.539654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14798e06-8dc9-4e3f-821b-3b4c7d663e87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:21.541103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14798e06-8dc9-4e3f-821b-3b4c7d663e87 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:21.547444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 71/145] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:21 2022] GET /identity/v3/role_assignments?user.id=5c36c5fbc8c04f3f8235a61fc5992850&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:22.228967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80b2ed-1fe6-4011-9f4a-c0bfa7a614f8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.229381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80b2ed-1fe6-4011-9f4a-c0bfa7a614f8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.229581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a80b2ed-1fe6-4011-9f4a-c0bfa7a614f8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.231068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 75/146] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:22.238221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82281fe-2f3b-4760-a862-8067db74571d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.238517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82281fe-2f3b-4760-a862-8067db74571d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.238780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82281fe-2f3b-4760-a862-8067db74571d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.258393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e82281fe-2f3b-4760-a862-8067db74571d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:22.259635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e82281fe-2f3b-4760-a862-8067db74571d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:22.328374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:22.340438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 72/147] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:22.377015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80b2ed-1fe6-4011-9f4a-c0bfa7a614f8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:22.390594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831540b-54be-4df1-b55c-6c4b51cf78c1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.390910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831540b-54be-4df1-b55c-6c4b51cf78c1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.391153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831540b-54be-4df1-b55c-6c4b51cf78c1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.392681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 73/148] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:22.411414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c64e63-a438-49e6-b4bc-e5fe3ef4943d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.411741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c64e63-a438-49e6-b4bc-e5fe3ef4943d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.411974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c64e63-a438-49e6-b4bc-e5fe3ef4943d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.454142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e9c64e63-a438-49e6-b4bc-e5fe3ef4943d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:22.455396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e9c64e63-a438-49e6-b4bc-e5fe3ef4943d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:22.520604 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a80b2ed-1fe6-4011-9f4a-c0bfa7a614f8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:22.527985 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:22.555971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0d1c8-4034-4d41-920d-abf740a12318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:22.556636 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0d1c8-4034-4d41-920d-abf740a12318 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.556924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0d1c8-4034-4d41-920d-abf740a12318 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.557207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0d1c8-4034-4d41-920d-abf740a12318 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.574525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0d1c8-4034-4d41-920d-abf740a12318 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-15f0d1c8-4034-4d41-920d-abf740a12318 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:22.579574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:22.580445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:22.581317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:22.586725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 76/149] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:11:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:22.592931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 74/150] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 26 07:11:22.593835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0d1c8-4034-4d41-920d-abf740a12318 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:22.622798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c64e63-a438-49e6-b4bc-e5fe3ef4943d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:22.716074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0d1c8-4034-4d41-920d-abf740a12318 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:22.783003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4828e8-cdb5-490a-bb00-cf80f0cc0594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:22.783564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4828e8-cdb5-490a-bb00-cf80f0cc0594 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.783849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4828e8-cdb5-490a-bb00-cf80f0cc0594 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.784167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4828e8-cdb5-490a-bb00-cf80f0cc0594 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.784905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4828e8-cdb5-490a-bb00-cf80f0cc0594 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:22.785215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4828e8-cdb5-490a-bb00-cf80f0cc0594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:22.787966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4828e8-cdb5-490a-bb00-cf80f0cc0594 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:22.805872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 77/151] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:22.812379 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4828e8-cdb5-490a-bb00-cf80f0cc0594 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:22.829514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c64e63-a438-49e6-b4bc-e5fe3ef4943d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:22.850041 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:22.860477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:22.874375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7225b1-33b2-4d7c-95b9-c8b9e14b7bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:22.875041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7225b1-33b2-4d7c-95b9-c8b9e14b7bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.875453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7225b1-33b2-4d7c-95b9-c8b9e14b7bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.875672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7225b1-33b2-4d7c-95b9-c8b9e14b7bc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.887506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7225b1-33b2-4d7c-95b9-c8b9e14b7bc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8d7225b1-33b2-4d7c-95b9-c8b9e14b7bc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:22.892552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:22.893532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:22.894580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 75/152] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:22.895772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb717ea1-794a-49eb-a803-cc466b9c0f5b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.895772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb717ea1-794a-49eb-a803-cc466b9c0f5b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.895920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb717ea1-794a-49eb-a803-cc466b9c0f5b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.897668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 76/153] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:22.902710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9fa7d-01ae-40f4-a256-58674400818d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.903001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9fa7d-01ae-40f4-a256-58674400818d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.903276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb9fa7d-01ae-40f4-a256-58674400818d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.908759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 77/154] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:22.908759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9fa7d-01ae-40f4-a256-58674400818d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:22.937804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4828e8-cdb5-490a-bb00-cf80f0cc0594 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:22.972960 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132b204-4da1-4ee8-b86a-e16084017f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:22.973613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132b204-4da1-4ee8-b86a-e16084017f77 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.973929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132b204-4da1-4ee8-b86a-e16084017f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.974240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132b204-4da1-4ee8-b86a-e16084017f77 None admin] PATH_INFO: `/v3/users/9e725435b0294296a28451de9a0d08c4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.975419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132b204-4da1-4ee8-b86a-e16084017f77 None admin] RBAC: Authorizing `identity:get_user(user_id=9e725435b0294296a28451de9a0d08c4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:22.976872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7132b204-4da1-4ee8-b86a-e16084017f77 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:22.978534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 78/155] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:22 2022] GET /identity/v3/users/9e725435b0294296a28451de9a0d08c4 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:22.979651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79629a6e-8127-4a15-91c0-c99f7ea6e21a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:22.979936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79629a6e-8127-4a15-91c0-c99f7ea6e21a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:22.980215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79629a6e-8127-4a15-91c0-c99f7ea6e21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:22.997867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-79629a6e-8127-4a15-91c0-c99f7ea6e21a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:22.999053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-79629a6e-8127-4a15-91c0-c99f7ea6e21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:23.018386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb9fa7d-01ae-40f4-a256-58674400818d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:23.053719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552c2e2-3c00-43c0-8693-59e0e278c831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:23.054317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552c2e2-3c00-43c0-8693-59e0e278c831 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.054600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552c2e2-3c00-43c0-8693-59e0e278c831 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.054871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552c2e2-3c00-43c0-8693-59e0e278c831 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.055651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552c2e2-3c00-43c0-8693-59e0e278c831 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:23.055941 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552c2e2-3c00-43c0-8693-59e0e278c831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:23.057405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552c2e2-3c00-43c0-8693-59e0e278c831 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:23.065499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 78/156] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:23.066735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da6232-08d3-4bfb-898a-d13fec8abc5c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.067015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da6232-08d3-4bfb-898a-d13fec8abc5c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.067370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da6232-08d3-4bfb-898a-d13fec8abc5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.081988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 79/157] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:23.081988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ff0bc-accd-4134-98ec-3625b12ce8ff None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.081988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ff0bc-accd-4134-98ec-3625b12ce8ff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.081988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327ff0bc-accd-4134-98ec-3625b12ce8ff None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.081988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 80/158] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:23.081988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ff0bc-accd-4134-98ec-3625b12ce8ff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:23.098739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b3da6232-08d3-4bfb-898a-d13fec8abc5c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:23.099925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b3da6232-08d3-4bfb-898a-d13fec8abc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:23.103105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:23.138504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:23.166951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 79/159] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:23.168943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da6232-08d3-4bfb-898a-d13fec8abc5c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:23.181499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327ff0bc-accd-4134-98ec-3625b12ce8ff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:23.216889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f57e24-dc04-415c-a688-61179d8f98a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:23.217514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57e24-dc04-415c-a688-61179d8f98a9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.217801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57e24-dc04-415c-a688-61179d8f98a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.218095 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f57e24-dc04-415c-a688-61179d8f98a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.270971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da6232-08d3-4bfb-898a-d13fec8abc5c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:23.287455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f57e24-dc04-415c-a688-61179d8f98a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:23.288945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f57e24-dc04-415c-a688-61179d8f98a9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:23.304569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae376ca-9a12-4097-b6ad-64ee8efc1b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:23.305114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae376ca-9a12-4097-b6ad-64ee8efc1b12 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.305386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae376ca-9a12-4097-b6ad-64ee8efc1b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.305652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae376ca-9a12-4097-b6ad-64ee8efc1b12 None admin] PATH_INFO: `/v3/users/5c36c5fbc8c04f3f8235a61fc5992850` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.306828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae376ca-9a12-4097-b6ad-64ee8efc1b12 None admin] RBAC: Authorizing `identity:get_user(user_id=5c36c5fbc8c04f3f8235a61fc5992850)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:23.308259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae376ca-9a12-4097-b6ad-64ee8efc1b12 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:23.309875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 80/160] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:23 2022] GET /identity/v3/users/5c36c5fbc8c04f3f8235a61fc5992850 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:23.310969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e9f1-118f-4d04-9ec2-77d0a86a7466 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.311250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e9f1-118f-4d04-9ec2-77d0a86a7466 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.311520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461e9f1-118f-4d04-9ec2-77d0a86a7466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.318241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:23.328763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0461e9f1-118f-4d04-9ec2-77d0a86a7466 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:23.329953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0461e9f1-118f-4d04-9ec2-77d0a86a7466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:23.365267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:23.393781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 81/161] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:23.395329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f57e24-dc04-415c-a688-61179d8f98a9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:23.402613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 81/162] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:23.406548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461e9f1-118f-4d04-9ec2-77d0a86a7466 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:23.503679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f57e24-dc04-415c-a688-61179d8f98a9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:23.512021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0461e9f1-118f-4d04-9ec2-77d0a86a7466 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:23.527068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:23.538926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755903dc-677d-48ef-abc3-2cf555aa8600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:23.539582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755903dc-677d-48ef-abc3-2cf555aa8600 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.539862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755903dc-677d-48ef-abc3-2cf555aa8600 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.540143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755903dc-677d-48ef-abc3-2cf555aa8600 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.547567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d49028-0c24-4ad8-b169-905561e8586b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:23.548215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d49028-0c24-4ad8-b169-905561e8586b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.548497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d49028-0c24-4ad8-b169-905561e8586b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.548771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d49028-0c24-4ad8-b169-905561e8586b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.550206 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755903dc-677d-48ef-abc3-2cf555aa8600 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-755903dc-677d-48ef-abc3-2cf555aa8600 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:23.555304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:23.556894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 82/163] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:23.558433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755903dc-677d-48ef-abc3-2cf555aa8600 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:23.562720 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d49028-0c24-4ad8-b169-905561e8586b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b6d49028-0c24-4ad8-b169-905561e8586b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:23.567889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:23.569972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:23.571733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 82/164] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:23.574146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d49028-0c24-4ad8-b169-905561e8586b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:23.606242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:23.669533 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755903dc-677d-48ef-abc3-2cf555aa8600 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:23.676711 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d49028-0c24-4ad8-b169-905561e8586b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:23.707543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fed08-4062-48f6-b732-36561a688419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:23.708288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fed08-4062-48f6-b732-36561a688419 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.708570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fed08-4062-48f6-b732-36561a688419 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.708861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fed08-4062-48f6-b732-36561a688419 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.714252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9de192-a070-4cbf-8d8e-300c70fda8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:23.715021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9de192-a070-4cbf-8d8e-300c70fda8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.715382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9de192-a070-4cbf-8d8e-300c70fda8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.715744 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9de192-a070-4cbf-8d8e-300c70fda8c3 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.717035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9de192-a070-4cbf-8d8e-300c70fda8c3 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:23.718882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9de192-a070-4cbf-8d8e-300c70fda8c3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:23.720585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 83/165] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:23 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:23.721339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9de192-a070-4cbf-8d8e-300c70fda8c3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:23.768309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:23.784578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fed08-4062-48f6-b732-36561a688419 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:23.786099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fed08-4062-48f6-b732-36561a688419 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:23.819972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:23.823447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9de192-a070-4cbf-8d8e-300c70fda8c3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:23.859572 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9a8ab-3854-4f98-8a61-db890c9bcac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:23.860227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9a8ab-3854-4f98-8a61-db890c9bcac1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:23.860507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9a8ab-3854-4f98-8a61-db890c9bcac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:23.860779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9a8ab-3854-4f98-8a61-db890c9bcac1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:23.861982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9a8ab-3854-4f98-8a61-db890c9bcac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:23.863442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9a8ab-3854-4f98-8a61-db890c9bcac1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:23.865035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 84/166] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:23.865853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9a8ab-3854-4f98-8a61-db890c9bcac1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:23.896666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 83/167] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:23.897506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fed08-4062-48f6-b732-36561a688419 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:23.966479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9a8ab-3854-4f98-8a61-db890c9bcac1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:23.985595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:24.001684 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535073d-3de9-4702-94e7-019112942fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:24.002353 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535073d-3de9-4702-94e7-019112942fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:24.002636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535073d-3de9-4702-94e7-019112942fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:24.002970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535073d-3de9-4702-94e7-019112942fd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:24.003694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535073d-3de9-4702-94e7-019112942fd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:24.004034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535073d-3de9-4702-94e7-019112942fd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:24.004814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fed08-4062-48f6-b732-36561a688419 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:24.005442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e535073d-3de9-4702-94e7-019112942fd3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:24.013610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 85/168] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:24.014533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535073d-3de9-4702-94e7-019112942fd3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:24.046232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb507b-b1c7-4b54-bfd3-bf631008de85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:24.046872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb507b-b1c7-4b54-bfd3-bf631008de85 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:24.047157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb507b-b1c7-4b54-bfd3-bf631008de85 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:24.047432 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb507b-b1c7-4b54-bfd3-bf631008de85 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:24.048159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb507b-b1c7-4b54-bfd3-bf631008de85 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:24.048465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb507b-b1c7-4b54-bfd3-bf631008de85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:24.049989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb507b-b1c7-4b54-bfd3-bf631008de85 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:24.052572 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:24.059996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 84/169] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:24.065660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb507b-b1c7-4b54-bfd3-bf631008de85 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:24.123179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e535073d-3de9-4702-94e7-019112942fd3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:24.158994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12bbd2-b3f4-4e79-a5fe-05226dfa462b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:24.159618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12bbd2-b3f4-4e79-a5fe-05226dfa462b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:24.159903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12bbd2-b3f4-4e79-a5fe-05226dfa462b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:24.160167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b12bbd2-b3f4-4e79-a5fe-05226dfa462b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:24.160960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12bbd2-b3f4-4e79-a5fe-05226dfa462b None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=9ad9190f63064d308a0423e34054b7df, user.id=9e725435b0294296a28451de9a0d08c4) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:24.161267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12bbd2-b3f4-4e79-a5fe-05226dfa462b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:24.162745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b12bbd2-b3f4-4e79-a5fe-05226dfa462b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:24.166236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb507b-b1c7-4b54-bfd3-bf631008de85 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:24.168512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 86/170] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:24 2022] GET /identity/v3/role_assignments?user.id=9e725435b0294296a28451de9a0d08c4&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:24.169385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12bbd2-b3f4-4e79-a5fe-05226dfa462b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:24.221672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d0af1-d515-49b9-bf33-7e242b2304a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:24.222834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d0af1-d515-49b9-bf33-7e242b2304a9 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:24.223304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d0af1-d515-49b9-bf33-7e242b2304a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:24.223712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d0af1-d515-49b9-bf33-7e242b2304a9 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:24.224751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d0af1-d515-49b9-bf33-7e242b2304a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:24.227389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2d0af1-d515-49b9-bf33-7e242b2304a9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:24.260721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 85/171] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:11:24 2022] POST /identity/v3/users => generated 283 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:11:24.262139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d0af1-d515-49b9-bf33-7e242b2304a9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:24.266543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:24.293524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:24.302878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b12bbd2-b3f4-4e79-a5fe-05226dfa462b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:24.371973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f61495-c0f2-4d1e-abe2-4fd0403d2820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:24.372930 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f61495-c0f2-4d1e-abe2-4fd0403d2820 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:24.373400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f61495-c0f2-4d1e-abe2-4fd0403d2820 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:24.373791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f61495-c0f2-4d1e-abe2-4fd0403d2820 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:24.376764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f61495-c0f2-4d1e-abe2-4fd0403d2820 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:24.380839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f61495-c0f2-4d1e-abe2-4fd0403d2820 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:24.385473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 87/172] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:24 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:24.387738 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f61495-c0f2-4d1e-abe2-4fd0403d2820 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:24.405441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d0af1-d515-49b9-bf33-7e242b2304a9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:24.456737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa811d-1ff8-4ea6-87d5-d7160a957438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:24.457425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa811d-1ff8-4ea6-87d5-d7160a957438 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:24.457715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa811d-1ff8-4ea6-87d5-d7160a957438 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:24.458040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa811d-1ff8-4ea6-87d5-d7160a957438 None admin] PATH_INFO: `/v3/users/5c36c5fbc8c04f3f8235a61fc5992850` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:24.459282 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa811d-1ff8-4ea6-87d5-d7160a957438 None admin] RBAC: Authorizing `identity:get_user(user_id=5c36c5fbc8c04f3f8235a61fc5992850)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:24.460751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaa811d-1ff8-4ea6-87d5-d7160a957438 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:24.462622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 86/173] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:24 2022] GET /identity/v3/users/5c36c5fbc8c04f3f8235a61fc5992850 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:24.463297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa811d-1ff8-4ea6-87d5-d7160a957438 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:24.562798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:24.579353 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f61495-c0f2-4d1e-abe2-4fd0403d2820 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:24.614016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa811d-1ff8-4ea6-87d5-d7160a957438 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:24.628148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f6705-e660-4f97-9a6e-dad23129f468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:24.629941 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f6705-e660-4f97-9a6e-dad23129f468 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:24.629941 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f6705-e660-4f97-9a6e-dad23129f468 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:24.630199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f6705-e660-4f97-9a6e-dad23129f468 None admin] PATH_INFO: `/v3/users/55d6f7dcb65a491c9baea3c593780f8a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:24.633397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f6705-e660-4f97-9a6e-dad23129f468 None admin] RBAC: Authorizing `identity:get_user(user_id=55d6f7dcb65a491c9baea3c593780f8a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:24.635687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f6705-e660-4f97-9a6e-dad23129f468 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:24.637424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 88/174] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:24 2022] GET /identity/v3/users/55d6f7dcb65a491c9baea3c593780f8a => generated 289 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:24.639938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f6705-e660-4f97-9a6e-dad23129f468 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:24.658263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af21a9-094a-492d-8b8d-000506636e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:24.658905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af21a9-094a-492d-8b8d-000506636e7f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:24.659257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af21a9-094a-492d-8b8d-000506636e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:24.659523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af21a9-094a-492d-8b8d-000506636e7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:24.660397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af21a9-094a-492d-8b8d-000506636e7f None admin] RBAC: Adding query filter params (role.id=5e0c08646046426385776839f1f168cd, scope.project.id=9ad9190f63064d308a0423e34054b7df, user.id=5c36c5fbc8c04f3f8235a61fc5992850) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:24.660694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af21a9-094a-492d-8b8d-000506636e7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:24.662320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af21a9-094a-492d-8b8d-000506636e7f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:24.669190 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 87/175] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:24 2022] GET /identity/v3/role_assignments?user.id=5c36c5fbc8c04f3f8235a61fc5992850&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=5e0c08646046426385776839f1f168cd => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:24.670136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af21a9-094a-492d-8b8d-000506636e7f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:24.706301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:24.819460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f6705-e660-4f97-9a6e-dad23129f468 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:24.828183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af21a9-094a-492d-8b8d-000506636e7f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:24.858966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408195f-dc15-47a7-bb91-a58f4200c806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:24.859616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408195f-dc15-47a7-bb91-a58f4200c806 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:24.859892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408195f-dc15-47a7-bb91-a58f4200c806 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:24.860164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3408195f-dc15-47a7-bb91-a58f4200c806 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:24.869790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7411450-4f87-451e-8998-95bc48567dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:24.871476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7411450-4f87-451e-8998-95bc48567dcd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:24.871758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7411450-4f87-451e-8998-95bc48567dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:24.872032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7411450-4f87-451e-8998-95bc48567dcd None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:24.873337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7411450-4f87-451e-8998-95bc48567dcd None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:24.874943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7411450-4f87-451e-8998-95bc48567dcd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:24.876725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 88/176] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:24 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:24.884182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7411450-4f87-451e-8998-95bc48567dcd None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:24.888241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:24.941231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408195f-dc15-47a7-bb91-a58f4200c806 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:24.943186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3408195f-dc15-47a7-bb91-a58f4200c806 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:24.990028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:25.014552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7411450-4f87-451e-8998-95bc48567dcd None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:25.056061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebba9f2-18f6-48af-a5d5-613254a433e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:25.056755 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebba9f2-18f6-48af-a5d5-613254a433e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:25.057037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebba9f2-18f6-48af-a5d5-613254a433e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:25.057312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebba9f2-18f6-48af-a5d5-613254a433e6 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df/users/55d6f7dcb65a491c9baea3c593780f8a/roles/511279b616c245e2baad4d3a59cd477e` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:25.059356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebba9f2-18f6-48af-a5d5-613254a433e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ad9190f63064d308a0423e34054b7df, user_id=55d6f7dcb65a491c9baea3c593780f8a, role_id=511279b616c245e2baad4d3a59cd477e)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:25.061360 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebba9f2-18f6-48af-a5d5-613254a433e6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:25.082164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 89/177] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:11:24 2022] PUT /identity/v3/projects/9ad9190f63064d308a0423e34054b7df/users/55d6f7dcb65a491c9baea3c593780f8a/roles/511279b616c245e2baad4d3a59cd477e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:25.094127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 89/178] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:25.100015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebba9f2-18f6-48af-a5d5-613254a433e6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:25.187005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:25.234603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebba9f2-18f6-48af-a5d5-613254a433e6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:25.284946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbc865-4901-4972-836d-d7878e81a57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:25.285620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbc865-4901-4972-836d-d7878e81a57d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:25.285935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbc865-4901-4972-836d-d7878e81a57d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:25.286208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbc865-4901-4972-836d-d7878e81a57d None admin] PATH_INFO: `/v3/projects/9af2f393366b4f8b8299ac7d763b021a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:25.287425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbc865-4901-4972-836d-d7878e81a57d None admin] RBAC: Authorizing `identity:get_project(project_id=9af2f393366b4f8b8299ac7d763b021a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:25.288883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edbc865-4901-4972-836d-d7878e81a57d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:25.290653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 90/179] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:25 2022] GET /identity/v3/projects/9af2f393366b4f8b8299ac7d763b021a => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:25.296804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408195f-dc15-47a7-bb91-a58f4200c806 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:25.419914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3408195f-dc15-47a7-bb91-a58f4200c806 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:25.454134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:25.460507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116a180-ebf0-47dc-9e9f-f58b79d09082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:25.461152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116a180-ebf0-47dc-9e9f-f58b79d09082 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:25.461434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116a180-ebf0-47dc-9e9f-f58b79d09082 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:25.461707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116a180-ebf0-47dc-9e9f-f58b79d09082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:25.462616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116a180-ebf0-47dc-9e9f-f58b79d09082 None admin] RBAC: Adding query filter params (role.id=64dfc66a8abe4468ac4f1ca071d28bd4, scope.project.id=9af2f393366b4f8b8299ac7d763b021a, user.id=5c36c5fbc8c04f3f8235a61fc5992850) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:25.463293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116a180-ebf0-47dc-9e9f-f58b79d09082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:25.465001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8116a180-ebf0-47dc-9e9f-f58b79d09082 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:25.472239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 90/180] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:25 2022] GET /identity/v3/role_assignments?user.id=5c36c5fbc8c04f3f8235a61fc5992850&scope.project.id=9af2f393366b4f8b8299ac7d763b021a&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:26.728911 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2faaa-dae9-4560-aee3-3699eec4f35b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:26.729406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2faaa-dae9-4560-aee3-3699eec4f35b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:26.729406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2faaa-dae9-4560-aee3-3699eec4f35b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:26.730947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 91/181] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:26.736736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2b9cc-ade8-40ae-be84-62ab9e04406e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:26.737269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2b9cc-ade8-40ae-be84-62ab9e04406e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:26.737340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d2b9cc-ade8-40ae-be84-62ab9e04406e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:26.756454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-22d2b9cc-ade8-40ae-be84-62ab9e04406e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:26.757662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-22d2b9cc-ade8-40ae-be84-62ab9e04406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:26.830370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 91/182] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:26.853598 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2faaa-dae9-4560-aee3-3699eec4f35b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:26.962472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2faaa-dae9-4560-aee3-3699eec4f35b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:26.974252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a67640-99c2-4e14-b6f4-cede0e574f5d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:26.974560 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a67640-99c2-4e14-b6f4-cede0e574f5d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:26.974830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a67640-99c2-4e14-b6f4-cede0e574f5d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:26.977067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 92/183] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:26.982807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ded054-9b02-4cf5-92e4-e29d4a8befe3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:26.983392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ded054-9b02-4cf5-92e4-e29d4a8befe3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:26.983674 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ded054-9b02-4cf5-92e4-e29d4a8befe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:26.997479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:27.002674 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-31ded054-9b02-4cf5-92e4-e29d4a8befe3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:27.003942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-31ded054-9b02-4cf5-92e4-e29d4a8befe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:27.004188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dfbc6-7581-409c-870c-1ba5f566ae16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:27.004863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dfbc6-7581-409c-870c-1ba5f566ae16 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.005143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dfbc6-7581-409c-870c-1ba5f566ae16 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.005422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331dfbc6-7581-409c-870c-1ba5f566ae16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.016975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331dfbc6-7581-409c-870c-1ba5f566ae16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-331dfbc6-7581-409c-870c-1ba5f566ae16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:27.022609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:27.023486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:27.024275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 92/184] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:11:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:27.031411 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dfbc6-7581-409c-870c-1ba5f566ae16 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:27.079575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 93/185] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:27.102677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ded054-9b02-4cf5-92e4-e29d4a8befe3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:27.143947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331dfbc6-7581-409c-870c-1ba5f566ae16 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:27.160545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:27.181769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873d3185-7452-4ef1-8eb7-6d3eff6db5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:27.182530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3185-7452-4ef1-8eb7-6d3eff6db5bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.182811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3185-7452-4ef1-8eb7-6d3eff6db5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.183085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873d3185-7452-4ef1-8eb7-6d3eff6db5bc None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.183812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d3185-7452-4ef1-8eb7-6d3eff6db5bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:27.184446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d3185-7452-4ef1-8eb7-6d3eff6db5bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:27.185803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873d3185-7452-4ef1-8eb7-6d3eff6db5bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:27.195433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 93/186] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:27.197882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8da5c-7368-41aa-9fb1-22fa4af0088f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.198749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8da5c-7368-41aa-9fb1-22fa4af0088f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.199507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8da5c-7368-41aa-9fb1-22fa4af0088f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.205061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 94/187] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:27.205061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8da5c-7368-41aa-9fb1-22fa4af0088f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:27.207909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ded054-9b02-4cf5-92e4-e29d4a8befe3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:27.244267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90ed6c-dd10-4c65-bd71-c7c9530e62dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:27.245293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90ed6c-dd10-4c65-bd71-c7c9530e62dc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.245709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90ed6c-dd10-4c65-bd71-c7c9530e62dc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.246144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90ed6c-dd10-4c65-bd71-c7c9530e62dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.259714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90ed6c-dd10-4c65-bd71-c7c9530e62dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:27.264688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-0b90ed6c-dd10-4c65-bd71-c7c9530e62dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:27.266261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:27.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 94/188] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:27.273548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37c5e-a753-4127-8322-fff27d9d402a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.274293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37c5e-a753-4127-8322-fff27d9d402a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.274293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc37c5e-a753-4127-8322-fff27d9d402a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.291165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6fc37c5e-a753-4127-8322-fff27d9d402a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:27.292367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6fc37c5e-a753-4127-8322-fff27d9d402a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:27.324262 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8da5c-7368-41aa-9fb1-22fa4af0088f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:27.362999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 95/189] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:27.367515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd54d6-0067-4d5f-8f9c-753c1be035d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:27.368431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd54d6-0067-4d5f-8f9c-753c1be035d8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.368888 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd54d6-0067-4d5f-8f9c-753c1be035d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.369394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd54d6-0067-4d5f-8f9c-753c1be035d8 None admin] PATH_INFO: `/v3/users/99464bba28ad437f8750d6fbc46b02ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.374650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc37c5e-a753-4127-8322-fff27d9d402a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:27.394057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd54d6-0067-4d5f-8f9c-753c1be035d8 None admin] RBAC: Authorizing `identity:get_user(user_id=99464bba28ad437f8750d6fbc46b02ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:27.396320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd54d6-0067-4d5f-8f9c-753c1be035d8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:27.399106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 95/190] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:27 2022] GET /identity/v3/users/99464bba28ad437f8750d6fbc46b02ae => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:27.400204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd54d6-0067-4d5f-8f9c-753c1be035d8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:27.442467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:27.498305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc37c5e-a753-4127-8322-fff27d9d402a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:27.513290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd54d6-0067-4d5f-8f9c-753c1be035d8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:27.525286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:27.531355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7b3d4-9056-4771-86ca-c468d25ccacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:27.531987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7b3d4-9056-4771-86ca-c468d25ccacf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.532267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7b3d4-9056-4771-86ca-c468d25ccacf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.532537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7b3d4-9056-4771-86ca-c468d25ccacf None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.533260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7b3d4-9056-4771-86ca-c468d25ccacf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:27.533561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7b3d4-9056-4771-86ca-c468d25ccacf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:27.535051 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d7b3d4-9056-4771-86ca-c468d25ccacf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:27.543022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 96/191] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:27.546659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7b3d4-9056-4771-86ca-c468d25ccacf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:27.550001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27de13-f4d7-4d84-a32b-7f3811c5ec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:27.550659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27de13-f4d7-4d84-a32b-7f3811c5ec69 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.550942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27de13-f4d7-4d84-a32b-7f3811c5ec69 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.551215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27de13-f4d7-4d84-a32b-7f3811c5ec69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.552412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27de13-f4d7-4d84-a32b-7f3811c5ec69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:27.553906 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27de13-f4d7-4d84-a32b-7f3811c5ec69 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:27.555529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 96/192] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:27.556673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eba99e-d69c-4786-bfe9-e08e48cebd9e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.556955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eba99e-d69c-4786-bfe9-e08e48cebd9e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.557227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eba99e-d69c-4786-bfe9-e08e48cebd9e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.558644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 97/193] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:27.559172 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eba99e-d69c-4786-bfe9-e08e48cebd9e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:27.652036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7b3d4-9056-4771-86ca-c468d25ccacf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:27.658914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:27.663660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eba99e-d69c-4786-bfe9-e08e48cebd9e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:27.685168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e67014-9633-4a93-821f-7132fe2e421f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:27.685837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e67014-9633-4a93-821f-7132fe2e421f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.686120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e67014-9633-4a93-821f-7132fe2e421f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.686392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e67014-9633-4a93-821f-7132fe2e421f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.699424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b863e-aa4e-40da-9bf8-f46fad3e8312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:27.700082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b863e-aa4e-40da-9bf8-f46fad3e8312 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.700385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b863e-aa4e-40da-9bf8-f46fad3e8312 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.700660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214b863e-aa4e-40da-9bf8-f46fad3e8312 None admin] PATH_INFO: `/v3/users/99464bba28ad437f8750d6fbc46b02ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.701872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b863e-aa4e-40da-9bf8-f46fad3e8312 None admin] RBAC: Authorizing `identity:get_user(user_id=99464bba28ad437f8750d6fbc46b02ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:27.703605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214b863e-aa4e-40da-9bf8-f46fad3e8312 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:27.705403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 98/194] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:27 2022] GET /identity/v3/users/99464bba28ad437f8750d6fbc46b02ae => generated 283 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:27.706035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b863e-aa4e-40da-9bf8-f46fad3e8312 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:27.754972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:27.764097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e67014-9633-4a93-821f-7132fe2e421f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:27.765583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e67014-9633-4a93-821f-7132fe2e421f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:27.812307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214b863e-aa4e-40da-9bf8-f46fad3e8312 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:27.859137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49779d02-eb9d-4326-908a-0a53dc344173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:27.859804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779d02-eb9d-4326-908a-0a53dc344173 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.860077 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779d02-eb9d-4326-908a-0a53dc344173 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.860348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49779d02-eb9d-4326-908a-0a53dc344173 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.861064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49779d02-eb9d-4326-908a-0a53dc344173 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:27.862629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49779d02-eb9d-4326-908a-0a53dc344173 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:27.865362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 97/195] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:27.866683 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae78896-7826-446c-91c8-a2ba70a81b3a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:27.866973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae78896-7826-446c-91c8-a2ba70a81b3a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:27.867317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae78896-7826-446c-91c8-a2ba70a81b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:27.879535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:27.885938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5ae78896-7826-446c-91c8-a2ba70a81b3a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:27.887271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5ae78896-7826-446c-91c8-a2ba70a81b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:27.887461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 99/196] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:11:27 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:11:27.894803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49779d02-eb9d-4326-908a-0a53dc344173 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:27.987920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 98/197] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:27.993661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae78896-7826-446c-91c8-a2ba70a81b3a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:28.026069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49779d02-eb9d-4326-908a-0a53dc344173 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:28.028754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:28.069514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a57e5-0531-4dd5-a479-36237945764b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:28.070586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a57e5-0531-4dd5-a479-36237945764b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:28.070873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a57e5-0531-4dd5-a479-36237945764b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:28.071152 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a57e5-0531-4dd5-a479-36237945764b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:28.156402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a57e5-0531-4dd5-a479-36237945764b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:28.158685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a57e5-0531-4dd5-a479-36237945764b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:28.169858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae78896-7826-446c-91c8-a2ba70a81b3a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:28.210557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c59c4-411b-4c90-8e66-6c8cefbe2c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:28.211241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c59c4-411b-4c90-8e66-6c8cefbe2c34 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:28.211525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c59c4-411b-4c90-8e66-6c8cefbe2c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:28.211835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c59c4-411b-4c90-8e66-6c8cefbe2c34 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:28.213522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c59c4-411b-4c90-8e66-6c8cefbe2c34 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:28.215298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c59c4-411b-4c90-8e66-6c8cefbe2c34 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:28.217017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 99/198] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:27 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:28.218032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c59c4-411b-4c90-8e66-6c8cefbe2c34 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:28.222335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:28.281402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 100/199] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:28.283352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a57e5-0531-4dd5-a479-36237945764b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:28.286696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:28.344723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c59c4-411b-4c90-8e66-6c8cefbe2c34 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:28.382591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe70bd-23c3-42c0-975b-30540db47752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:28.383583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe70bd-23c3-42c0-975b-30540db47752 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:28.384055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe70bd-23c3-42c0-975b-30540db47752 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:28.384480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe70bd-23c3-42c0-975b-30540db47752 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:28.397985 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe70bd-23c3-42c0-975b-30540db47752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cbbe70bd-23c3-42c0-975b-30540db47752 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:28.403437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:28.404370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:28.405185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:28.412317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 100/200] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:28.413422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe70bd-23c3-42c0-975b-30540db47752 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:28.431360 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a57e5-0531-4dd5-a479-36237945764b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:28.474801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad9c17-a2ec-43da-8a6b-4896267d1dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:28.475755 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad9c17-a2ec-43da-8a6b-4896267d1dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:28.476213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad9c17-a2ec-43da-8a6b-4896267d1dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:28.476715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad9c17-a2ec-43da-8a6b-4896267d1dd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:28.477720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad9c17-a2ec-43da-8a6b-4896267d1dd0 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f, user.id=99464bba28ad437f8750d6fbc46b02ae) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:28.478299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad9c17-a2ec-43da-8a6b-4896267d1dd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:28.480058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad9c17-a2ec-43da-8a6b-4896267d1dd0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:28.487187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 101/201] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:28 2022] GET /identity/v3/role_assignments?user.id=99464bba28ad437f8750d6fbc46b02ae&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:28.488376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad9c17-a2ec-43da-8a6b-4896267d1dd0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:28.496932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:28.570823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe70bd-23c3-42c0-975b-30540db47752 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:28.624536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:28.644899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112e3c4-5ef2-426c-a982-e4580f1cb902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:28.645570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112e3c4-5ef2-426c-a982-e4580f1cb902 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:28.645897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112e3c4-5ef2-426c-a982-e4580f1cb902 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:28.646173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9112e3c4-5ef2-426c-a982-e4580f1cb902 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:28.647563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112e3c4-5ef2-426c-a982-e4580f1cb902 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:28.649053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9112e3c4-5ef2-426c-a982-e4580f1cb902 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:28.650816 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 101/202] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:28 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:28.657939 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112e3c4-5ef2-426c-a982-e4580f1cb902 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:28.678947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad9c17-a2ec-43da-8a6b-4896267d1dd0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:28.738415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1e23e-f6a3-493d-8f25-6e4f4acbd650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:28.739065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1e23e-f6a3-493d-8f25-6e4f4acbd650 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:28.739445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1e23e-f6a3-493d-8f25-6e4f4acbd650 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:28.739725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1e23e-f6a3-493d-8f25-6e4f4acbd650 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:28.740564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1e23e-f6a3-493d-8f25-6e4f4acbd650 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:28.740870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1e23e-f6a3-493d-8f25-6e4f4acbd650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:28.742317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1e23e-f6a3-493d-8f25-6e4f4acbd650 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:28.753344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 102/203] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:28.761558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1e23e-f6a3-493d-8f25-6e4f4acbd650 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:28.816012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9112e3c4-5ef2-426c-a982-e4580f1cb902 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:28.884802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1e23e-f6a3-493d-8f25-6e4f4acbd650 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:28.894885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:28.902048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba42ee1-ac37-42dd-a53d-969dc3a97e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:28.902836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba42ee1-ac37-42dd-a53d-969dc3a97e68 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:28.903225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba42ee1-ac37-42dd-a53d-969dc3a97e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:28.903565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba42ee1-ac37-42dd-a53d-969dc3a97e68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:28.904503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba42ee1-ac37-42dd-a53d-969dc3a97e68 None admin] RBAC: Adding query filter params (role.id=5e0c08646046426385776839f1f168cd, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f, user.id=99464bba28ad437f8750d6fbc46b02ae) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:28.906941 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba42ee1-ac37-42dd-a53d-969dc3a97e68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:28.908432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba42ee1-ac37-42dd-a53d-969dc3a97e68 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:28.913433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:28.915812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 102/204] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:28 2022] GET /identity/v3/role_assignments?user.id=99464bba28ad437f8750d6fbc46b02ae&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=5e0c08646046426385776839f1f168cd => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:28.935729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82e6dc-e8d4-4607-86ba-be0ea6985859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:28.936429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82e6dc-e8d4-4607-86ba-be0ea6985859 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:28.936713 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82e6dc-e8d4-4607-86ba-be0ea6985859 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:28.936993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a82e6dc-e8d4-4607-86ba-be0ea6985859 None admin] PATH_INFO: `/v3/users/55d6f7dcb65a491c9baea3c593780f8a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:28.938250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82e6dc-e8d4-4607-86ba-be0ea6985859 None admin] RBAC: Authorizing `identity:get_user(user_id=55d6f7dcb65a491c9baea3c593780f8a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:28.939726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a82e6dc-e8d4-4607-86ba-be0ea6985859 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:28.941430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 103/205] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:28 2022] GET /identity/v3/users/55d6f7dcb65a491c9baea3c593780f8a => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:28.947768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba42ee1-ac37-42dd-a53d-969dc3a97e68 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:29.071087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba42ee1-ac37-42dd-a53d-969dc3a97e68 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:29.113549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f1b2c-987e-41fd-addd-db5e09cff6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:29.114259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f1b2c-987e-41fd-addd-db5e09cff6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:29.114721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f1b2c-987e-41fd-addd-db5e09cff6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:29.115008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f1b2c-987e-41fd-addd-db5e09cff6d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:29.201912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:29.212938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f1b2c-987e-41fd-addd-db5e09cff6d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:29.214782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f1b2c-987e-41fd-addd-db5e09cff6d2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:29.324148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 103/206] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:29.333772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82e6dc-e8d4-4607-86ba-be0ea6985859 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:29.484730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a82e6dc-e8d4-4607-86ba-be0ea6985859 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:29.518773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374b178-644b-418b-9c42-507d1f738643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:29.519427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b178-644b-418b-9c42-507d1f738643 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:29.519709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b178-644b-418b-9c42-507d1f738643 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:29.519991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374b178-644b-418b-9c42-507d1f738643 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:29.521167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374b178-644b-418b-9c42-507d1f738643 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:29.522640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374b178-644b-418b-9c42-507d1f738643 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:29.524325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 104/207] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:29 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:29.530491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f1b2c-987e-41fd-addd-db5e09cff6d2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:29.665637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:29.671922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f1b2c-987e-41fd-addd-db5e09cff6d2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:29.721358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee161a-29d5-434b-8ead-c69de0120384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:29.722365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee161a-29d5-434b-8ead-c69de0120384 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:29.722782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee161a-29d5-434b-8ead-c69de0120384 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:29.723191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee161a-29d5-434b-8ead-c69de0120384 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:29.724146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee161a-29d5-434b-8ead-c69de0120384 None admin] RBAC: Adding query filter params (role.id=511279b616c245e2baad4d3a59cd477e, scope.project.id=9ad9190f63064d308a0423e34054b7df, user.id=55d6f7dcb65a491c9baea3c593780f8a) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:29.724589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee161a-29d5-434b-8ead-c69de0120384 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:29.726206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee161a-29d5-434b-8ead-c69de0120384 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:29.732618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 104/208] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:29 2022] GET /identity/v3/role_assignments?user.id=55d6f7dcb65a491c9baea3c593780f8a&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=511279b616c245e2baad4d3a59cd477e => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:30.272049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef88929-3c2d-431e-bb28-994088ea3adb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.272609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef88929-3c2d-431e-bb28-994088ea3adb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.272609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef88929-3c2d-431e-bb28-994088ea3adb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.274304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 105/209] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:30.279489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84177e1-bdc5-4187-a8b6-381dd8aa38a4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.279886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84177e1-bdc5-4187-a8b6-381dd8aa38a4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.280075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84177e1-bdc5-4187-a8b6-381dd8aa38a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.305615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b84177e1-bdc5-4187-a8b6-381dd8aa38a4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:30.306817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b84177e1-bdc5-4187-a8b6-381dd8aa38a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:30.326121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0c667-9ff3-4593-9b73-0c060d7b8c69 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.326421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0c667-9ff3-4593-9b73-0c060d7b8c69 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.326687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e0c667-9ff3-4593-9b73-0c060d7b8c69 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.328208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 106/210] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:11:30.333357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b62dd-5e13-49c9-8e4e-8f97bbd5c857 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.333639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b62dd-5e13-49c9-8e4e-8f97bbd5c857 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.333948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b62dd-5e13-49c9-8e4e-8f97bbd5c857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.352828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0b6b62dd-5e13-49c9-8e4e-8f97bbd5c857 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:30.354175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b6b62dd-5e13-49c9-8e4e-8f97bbd5c857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:30.389500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 105/211] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:30.413709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84177e1-bdc5-4187-a8b6-381dd8aa38a4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:30.426784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:30.429478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 107/212] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:30.457681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b62dd-5e13-49c9-8e4e-8f97bbd5c857 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:30.523295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:30.539538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84177e1-bdc5-4187-a8b6-381dd8aa38a4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:30.578219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6f599-3c94-4a6d-b1fd-f8256d326981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:30.579249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6f599-3c94-4a6d-b1fd-f8256d326981 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.579749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6f599-3c94-4a6d-b1fd-f8256d326981 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.580203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6f599-3c94-4a6d-b1fd-f8256d326981 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.580746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b62dd-5e13-49c9-8e4e-8f97bbd5c857 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:30.591963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6f599-3c94-4a6d-b1fd-f8256d326981 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6fd6f599-3c94-4a6d-b1fd-f8256d326981 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:30.597729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:30.598609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:30.599433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:30.602078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 106/213] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:30.607697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6f599-3c94-4a6d-b1fd-f8256d326981 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:30.617200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ad81a-d9bf-4938-b0be-3eedd55122cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:30.617886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ad81a-d9bf-4938-b0be-3eedd55122cc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.618169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ad81a-d9bf-4938-b0be-3eedd55122cc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.618458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ad81a-d9bf-4938-b0be-3eedd55122cc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.619595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ad81a-d9bf-4938-b0be-3eedd55122cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:30.621176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ad81a-d9bf-4938-b0be-3eedd55122cc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:30.622839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 108/214] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:30.628617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ad81a-d9bf-4938-b0be-3eedd55122cc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:30.697096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:30.718439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6f599-3c94-4a6d-b1fd-f8256d326981 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:30.749718 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ad81a-d9bf-4938-b0be-3eedd55122cc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:30.770533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8599a1f-c47b-43ea-b518-26d4b3a9ab8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:30.772237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8599a1f-c47b-43ea-b518-26d4b3a9ab8b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.772815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8599a1f-c47b-43ea-b518-26d4b3a9ab8b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.773383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8599a1f-c47b-43ea-b518-26d4b3a9ab8b None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.774949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8599a1f-c47b-43ea-b518-26d4b3a9ab8b None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:30.775447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8599a1f-c47b-43ea-b518-26d4b3a9ab8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:30.777020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8599a1f-c47b-43ea-b518-26d4b3a9ab8b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:30.793773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f145d-dbb1-4b13-85eb-45365ddddaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:30.794304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f145d-dbb1-4b13-85eb-45365ddddaee None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.794495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 107/215] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:30.794634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f145d-dbb1-4b13-85eb-45365ddddaee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.795193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f145d-dbb1-4b13-85eb-45365ddddaee None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.795974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f145d-dbb1-4b13-85eb-45365ddddaee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:30.796475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8599a1f-c47b-43ea-b518-26d4b3a9ab8b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:30.797530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f145d-dbb1-4b13-85eb-45365ddddaee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:30.815338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:30.828894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 109/216] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:11:30 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:11:30.929897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8599a1f-c47b-43ea-b518-26d4b3a9ab8b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:30.969401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c3c35-c9f0-4947-b8d2-71314bb6d415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:30.970241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c3c35-c9f0-4947-b8d2-71314bb6d415 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.970570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c3c35-c9f0-4947-b8d2-71314bb6d415 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.970845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c3c35-c9f0-4947-b8d2-71314bb6d415 None admin] PATH_INFO: `/v3/users/146fa5e1195c42e3b43efa0f2fde1d65` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.973165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170aad9-4c55-4549-b0e1-68a8b55357f3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.973475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170aad9-4c55-4549-b0e1-68a8b55357f3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.973762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170aad9-4c55-4549-b0e1-68a8b55357f3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.975502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 110/217] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:11:30.981145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ec803-5366-46f8-b6ce-de2a39f7e0a4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:30.981467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ec803-5366-46f8-b6ce-de2a39f7e0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:30.981700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ec803-5366-46f8-b6ce-de2a39f7e0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:30.988276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c3c35-c9f0-4947-b8d2-71314bb6d415 None admin] RBAC: Authorizing `identity:get_user(user_id=146fa5e1195c42e3b43efa0f2fde1d65)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:30.989865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c3c35-c9f0-4947-b8d2-71314bb6d415 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:30.991989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 108/218] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:30 2022] GET /identity/v3/users/146fa5e1195c42e3b43efa0f2fde1d65 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:30.998861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c3c35-c9f0-4947-b8d2-71314bb6d415 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:31.004239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6c0ec803-5366-46f8-b6ce-de2a39f7e0a4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:31.005526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6c0ec803-5366-46f8-b6ce-de2a39f7e0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:31.084280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:31.098940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 111/219] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:31.125593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ec803-5366-46f8-b6ce-de2a39f7e0a4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:31.130244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c3c35-c9f0-4947-b8d2-71314bb6d415 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:31.180475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2ca55-a24e-41ed-9b59-575cd3d5859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:31.181134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2ca55-a24e-41ed-9b59-575cd3d5859d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:31.181423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2ca55-a24e-41ed-9b59-575cd3d5859d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:31.182226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2ca55-a24e-41ed-9b59-575cd3d5859d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:31.196753 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:31.266736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ec803-5366-46f8-b6ce-de2a39f7e0a4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:31.289822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2ca55-a24e-41ed-9b59-575cd3d5859d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:31.289822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d2ca55-a24e-41ed-9b59-575cd3d5859d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:31.316219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70140e-ea92-48e6-8df7-47b5ac4dd9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:31.317759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70140e-ea92-48e6-8df7-47b5ac4dd9b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:31.318473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70140e-ea92-48e6-8df7-47b5ac4dd9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:31.318887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70140e-ea92-48e6-8df7-47b5ac4dd9b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:31.333865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70140e-ea92-48e6-8df7-47b5ac4dd9b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cc70140e-ea92-48e6-8df7-47b5ac4dd9b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:31.340788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:31.341871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:31.342841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:31.345659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 112/220] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:11:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:31.359269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70140e-ea92-48e6-8df7-47b5ac4dd9b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:31.373205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:31.415343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 109/221] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:31.416787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdac28-3267-46f4-bf83-ddb3a05c2928 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:31.417083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdac28-3267-46f4-bf83-ddb3a05c2928 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:31.417361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdac28-3267-46f4-bf83-ddb3a05c2928 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:31.418899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 110/222] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:31.420789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdac28-3267-46f4-bf83-ddb3a05c2928 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:31.492854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70140e-ea92-48e6-8df7-47b5ac4dd9b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:31.503802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:31.530993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e677034-1e41-4d46-a6eb-a319991592a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:31.531737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677034-1e41-4d46-a6eb-a319991592a4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:31.532082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677034-1e41-4d46-a6eb-a319991592a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:31.532523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677034-1e41-4d46-a6eb-a319991592a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:31.533328 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e677034-1e41-4d46-a6eb-a319991592a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:31.533702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e677034-1e41-4d46-a6eb-a319991592a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:31.535280 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e677034-1e41-4d46-a6eb-a319991592a4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:31.544872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 113/223] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:31.551040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351059d7-f901-49ac-ae19-0daaf46ceeb3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:31.551399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351059d7-f901-49ac-ae19-0daaf46ceeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:31.551746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351059d7-f901-49ac-ae19-0daaf46ceeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:31.558792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdac28-3267-46f4-bf83-ddb3a05c2928 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:31.571199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-351059d7-f901-49ac-ae19-0daaf46ceeb3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:31.572611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-351059d7-f901-49ac-ae19-0daaf46ceeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:31.607760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7f169-cf17-4aa8-91bd-210fd45dd2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:31.609098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7f169-cf17-4aa8-91bd-210fd45dd2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:31.609545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7f169-cf17-4aa8-91bd-210fd45dd2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:31.609987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7f169-cf17-4aa8-91bd-210fd45dd2f1 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:31.611364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7f169-cf17-4aa8-91bd-210fd45dd2f1 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:31.613038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc7f169-cf17-4aa8-91bd-210fd45dd2f1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:31.614913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 111/224] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:31 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:31.620738 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7f169-cf17-4aa8-91bd-210fd45dd2f1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:31.651830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 114/225] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:31.653118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351059d7-f901-49ac-ae19-0daaf46ceeb3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:31.675108 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:31.760453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc7f169-cf17-4aa8-91bd-210fd45dd2f1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:31.778226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:31.782631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351059d7-f901-49ac-ae19-0daaf46ceeb3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:31.802782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b7436-18be-4cb7-bd85-203a8d00fcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:31.803781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7436-18be-4cb7-bd85-203a8d00fcfc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:31.804249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7436-18be-4cb7-bd85-203a8d00fcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:31.804689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b7436-18be-4cb7-bd85-203a8d00fcfc None admin] PATH_INFO: `/v3/users/99464bba28ad437f8750d6fbc46b02ae` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:31.806253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b7436-18be-4cb7-bd85-203a8d00fcfc None admin] RBAC: Authorizing `identity:get_user(user_id=99464bba28ad437f8750d6fbc46b02ae)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:31.808011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b7436-18be-4cb7-bd85-203a8d00fcfc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:31.810359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 112/226] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:31 2022] GET /identity/v3/users/99464bba28ad437f8750d6fbc46b02ae => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:31.811469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b7436-18be-4cb7-bd85-203a8d00fcfc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:31.828589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ba8a9-dbf5-437e-a1c2-69a54d0a65ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:31.829560 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ba8a9-dbf5-437e-a1c2-69a54d0a65ad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:31.830067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ba8a9-dbf5-437e-a1c2-69a54d0a65ad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:31.830558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ba8a9-dbf5-437e-a1c2-69a54d0a65ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:31.831725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ba8a9-dbf5-437e-a1c2-69a54d0a65ad None admin] RBAC: Adding query filter params (role.id=64dfc66a8abe4468ac4f1ca071d28bd4, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f, user.id=146fa5e1195c42e3b43efa0f2fde1d65) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:31.832229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ba8a9-dbf5-437e-a1c2-69a54d0a65ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:31.833945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ba8a9-dbf5-437e-a1c2-69a54d0a65ad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:31.840452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 115/227] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:31 2022] GET /identity/v3/role_assignments?user.id=146fa5e1195c42e3b43efa0f2fde1d65&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:31.841766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ba8a9-dbf5-437e-a1c2-69a54d0a65ad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:31.942876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b7436-18be-4cb7-bd85-203a8d00fcfc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:31.951954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:31.966955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ba8a9-dbf5-437e-a1c2-69a54d0a65ad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:31.982714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac76477-ab3c-4979-aeea-c09790e71495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:31.984618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac76477-ab3c-4979-aeea-c09790e71495 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:31.985311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac76477-ab3c-4979-aeea-c09790e71495 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:31.985958 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac76477-ab3c-4979-aeea-c09790e71495 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:31.998590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac76477-ab3c-4979-aeea-c09790e71495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:32.012188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae6bb2-1a5c-4b8b-8c7e-7296ec8cd3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:32.012188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae6bb2-1a5c-4b8b-8c7e-7296ec8cd3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:32.012188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae6bb2-1a5c-4b8b-8c7e-7296ec8cd3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:32.012188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae6bb2-1a5c-4b8b-8c7e-7296ec8cd3b5 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:32.012591 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae6bb2-1a5c-4b8b-8c7e-7296ec8cd3b5 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:32.012591 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae6bb2-1a5c-4b8b-8c7e-7296ec8cd3b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:32.012591 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 116/228] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:31 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5ac76477-ab3c-4979-aeea-c09790e71495 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:32.013561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:32.014424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:32.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:32.017770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 113/229] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:32.021642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae6bb2-1a5c-4b8b-8c7e-7296ec8cd3b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:32.031947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac76477-ab3c-4979-aeea-c09790e71495 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:32.058241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:32.137704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae6bb2-1a5c-4b8b-8c7e-7296ec8cd3b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:32.174177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7449b-75d1-4bc4-99b0-b1a7f547e14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:32.175293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7449b-75d1-4bc4-99b0-b1a7f547e14e None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:32.175994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7449b-75d1-4bc4-99b0-b1a7f547e14e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:32.176463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b7449b-75d1-4bc4-99b0-b1a7f547e14e None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/users/99464bba28ad437f8750d6fbc46b02ae/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:32.178759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7449b-75d1-4bc4-99b0-b1a7f547e14e None admin] RBAC: Authorizing `identity:create_grant(project_id=e07b3a0db08041fc87412eeb161bcf6f, user_id=99464bba28ad437f8750d6fbc46b02ae, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:32.180426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b7449b-75d1-4bc4-99b0-b1a7f547e14e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:32.210477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac76477-ab3c-4979-aeea-c09790e71495 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:32.216547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 117/230] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:11:32 2022] PUT /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/users/99464bba28ad437f8750d6fbc46b02ae/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:32.257585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f47117-0f24-4c1e-beae-8d53771ad82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:32.258318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f47117-0f24-4c1e-beae-8d53771ad82d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:32.258605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f47117-0f24-4c1e-beae-8d53771ad82d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:32.258885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f47117-0f24-4c1e-beae-8d53771ad82d None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:32.259654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f47117-0f24-4c1e-beae-8d53771ad82d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:32.259964 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f47117-0f24-4c1e-beae-8d53771ad82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:32.261417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f47117-0f24-4c1e-beae-8d53771ad82d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:32.276730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 114/231] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:32.282424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7449b-75d1-4bc4-99b0-b1a7f547e14e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:32.419031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:32.428413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b7449b-75d1-4bc4-99b0-b1a7f547e14e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:32.474185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f90c1f-8bf1-4382-9987-9a60029ec923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:32.474818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f90c1f-8bf1-4382-9987-9a60029ec923 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:32.475106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f90c1f-8bf1-4382-9987-9a60029ec923 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:32.475378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f90c1f-8bf1-4382-9987-9a60029ec923 None admin] PATH_INFO: `/v3/users/99464bba28ad437f8750d6fbc46b02ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:32.476569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f90c1f-8bf1-4382-9987-9a60029ec923 None admin] RBAC: Authorizing `identity:get_user(user_id=99464bba28ad437f8750d6fbc46b02ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:32.478059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f90c1f-8bf1-4382-9987-9a60029ec923 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:32.480017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 118/232] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:32 2022] GET /identity/v3/users/99464bba28ad437f8750d6fbc46b02ae => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:32.486883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f47117-0f24-4c1e-beae-8d53771ad82d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:32.528310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:32.633185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f47117-0f24-4c1e-beae-8d53771ad82d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:32.677682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaa86d-4c34-4940-8f0a-8c649a0ff285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:32.678374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa86d-4c34-4940-8f0a-8c649a0ff285 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:32.678659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa86d-4c34-4940-8f0a-8c649a0ff285 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:32.678935 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaa86d-4c34-4940-8f0a-8c649a0ff285 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:32.680148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaa86d-4c34-4940-8f0a-8c649a0ff285 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:32.681625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaa86d-4c34-4940-8f0a-8c649a0ff285 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:32.683661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 115/233] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:32 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:32.689503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f90c1f-8bf1-4382-9987-9a60029ec923 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:32.829486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f90c1f-8bf1-4382-9987-9a60029ec923 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:32.879529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61955bea-ffa7-4753-ab55-e5bb2ce6597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:32.880246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955bea-ffa7-4753-ab55-e5bb2ce6597f None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:32.880534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955bea-ffa7-4753-ab55-e5bb2ce6597f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:32.880813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61955bea-ffa7-4753-ab55-e5bb2ce6597f None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/users/99464bba28ad437f8750d6fbc46b02ae/roles/5e0c08646046426385776839f1f168cd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:32.883153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61955bea-ffa7-4753-ab55-e5bb2ce6597f None admin] RBAC: Authorizing `identity:create_grant(project_id=e07b3a0db08041fc87412eeb161bcf6f, user_id=99464bba28ad437f8750d6fbc46b02ae, role_id=5e0c08646046426385776839f1f168cd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:32.884614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61955bea-ffa7-4753-ab55-e5bb2ce6597f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:32.917169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 119/234] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:11:32 2022] PUT /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/users/99464bba28ad437f8750d6fbc46b02ae/roles/5e0c08646046426385776839f1f168cd => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:33.348365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ebccb-c0b5-4c77-81ef-455295215b60 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:33.348780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ebccb-c0b5-4c77-81ef-455295215b60 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:33.348967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68ebccb-c0b5-4c77-81ef-455295215b60 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:33.350638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 116/235] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:33.355747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e690158-4475-43e2-b9ec-b6cd2c4e606f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:33.356037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e690158-4475-43e2-b9ec-b6cd2c4e606f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:33.356316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e690158-4475-43e2-b9ec-b6cd2c4e606f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:33.378068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7e690158-4475-43e2-b9ec-b6cd2c4e606f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:33.379271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e690158-4475-43e2-b9ec-b6cd2c4e606f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:33.405468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:33.502430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 120/236] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:33.536897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ebccb-c0b5-4c77-81ef-455295215b60 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:33.668492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:33.679475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68ebccb-c0b5-4c77-81ef-455295215b60 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:33.730768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc30b7-c175-4eff-b966-797ab39c3a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:33.731430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc30b7-c175-4eff-b966-797ab39c3a16 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:33.731707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc30b7-c175-4eff-b966-797ab39c3a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:33.731973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc30b7-c175-4eff-b966-797ab39c3a16 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:33.743625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc30b7-c175-4eff-b966-797ab39c3a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3cbc30b7-c175-4eff-b966-797ab39c3a16 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:33.748970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:33.751165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:33.753295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 117/237] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:33.757781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e690158-4475-43e2-b9ec-b6cd2c4e606f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:33.878167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa15323-0a6c-4da3-b491-ee3b9267629c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:33.878542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa15323-0a6c-4da3-b491-ee3b9267629c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:33.878755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa15323-0a6c-4da3-b491-ee3b9267629c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:33.880249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 118/238] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:33.886465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dfe11-4c72-4d49-90d6-823c4fd8b6a4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:33.886741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dfe11-4c72-4d49-90d6-823c4fd8b6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:33.887009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2dfe11-4c72-4d49-90d6-823c4fd8b6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:33.909459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e690158-4475-43e2-b9ec-b6cd2c4e606f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:33.911891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ef2dfe11-4c72-4d49-90d6-823c4fd8b6a4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:33.913068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ef2dfe11-4c72-4d49-90d6-823c4fd8b6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:33.966527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f54e1-ebe8-4193-9554-ec2cc045d59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:33.967069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f54e1-ebe8-4193-9554-ec2cc045d59f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:33.967346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f54e1-ebe8-4193-9554-ec2cc045d59f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:33.967616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878f54e1-ebe8-4193-9554-ec2cc045d59f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:33.968355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f54e1-ebe8-4193-9554-ec2cc045d59f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:33.968974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f54e1-ebe8-4193-9554-ec2cc045d59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:33.971262 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878f54e1-ebe8-4193-9554-ec2cc045d59f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:33.990608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 121/239] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:33.996206 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f54e1-ebe8-4193-9554-ec2cc045d59f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:34.012527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:34.031751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 119/240] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:34.067584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dfe11-4c72-4d49-90d6-823c4fd8b6a4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:34.103797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878f54e1-ebe8-4193-9554-ec2cc045d59f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:34.140617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3719cc-f3da-449c-91f9-ad6608d70709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:34.141277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3719cc-f3da-449c-91f9-ad6608d70709 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.141555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3719cc-f3da-449c-91f9-ad6608d70709 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.141859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3719cc-f3da-449c-91f9-ad6608d70709 None admin] PATH_INFO: `/v3/users/1464bb3a65394de0914b2e5332b7af0b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.156632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3719cc-f3da-449c-91f9-ad6608d70709 None admin] RBAC: Authorizing `identity:get_user(user_id=1464bb3a65394de0914b2e5332b7af0b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:34.158143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3719cc-f3da-449c-91f9-ad6608d70709 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:34.160030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 122/241] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:33 2022] GET /identity/v3/users/1464bb3a65394de0914b2e5332b7af0b => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:34.165804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3719cc-f3da-449c-91f9-ad6608d70709 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:34.218247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:34.238953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:34.247592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2dfe11-4c72-4d49-90d6-823c4fd8b6a4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:34.271148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3719cc-f3da-449c-91f9-ad6608d70709 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:34.305606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018ef31-4a01-497b-98f7-f87433dc299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:34.306368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018ef31-4a01-497b-98f7-f87433dc299e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.306742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018ef31-4a01-497b-98f7-f87433dc299e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.307023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f018ef31-4a01-497b-98f7-f87433dc299e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.310287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bcf1c-f836-4b4b-946c-d1e0a420fd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:34.310951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bcf1c-f836-4b4b-946c-d1e0a420fd7a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.311234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bcf1c-f836-4b4b-946c-d1e0a420fd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.311510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155bcf1c-f836-4b4b-946c-d1e0a420fd7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.318576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bcf1c-f836-4b4b-946c-d1e0a420fd7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:34.320066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155bcf1c-f836-4b4b-946c-d1e0a420fd7a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:34.321762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 120/242] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:34.327090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bcf1c-f836-4b4b-946c-d1e0a420fd7a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:34.388125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018ef31-4a01-497b-98f7-f87433dc299e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:34.389589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f018ef31-4a01-497b-98f7-f87433dc299e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:34.470179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:34.507669 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 123/243] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 07:11:34.513780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018ef31-4a01-497b-98f7-f87433dc299e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:34.524387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155bcf1c-f836-4b4b-946c-d1e0a420fd7a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:34.572062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6ed86d-d433-484d-a413-7e965ccba9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:34.572960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ed86d-d433-484d-a413-7e965ccba9d7 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.573332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ed86d-d433-484d-a413-7e965ccba9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.573698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6ed86d-d433-484d-a413-7e965ccba9d7 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.574629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ed86d-d433-484d-a413-7e965ccba9d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:34.576295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6ed86d-d433-484d-a413-7e965ccba9d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:34.607356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 121/244] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:11:34 2022] POST /identity/v3/users => generated 293 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:11:34.609350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773a5d-a57e-4097-a4ec-ca3abe4d29fb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.609695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773a5d-a57e-4097-a4ec-ca3abe4d29fb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.610029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05773a5d-a57e-4097-a4ec-ca3abe4d29fb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.614892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f018ef31-4a01-497b-98f7-f87433dc299e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:34.615802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 122/245] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:34.617424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081f3a6-cfba-4296-a052-5073c1ca79f2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.617873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081f3a6-cfba-4296-a052-5073c1ca79f2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.618153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081f3a6-cfba-4296-a052-5073c1ca79f2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.619586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 123/246] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:34.621425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e621893-4fb5-44c4-954e-78e2ce5191a4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.621702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e621893-4fb5-44c4-954e-78e2ce5191a4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.621998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e621893-4fb5-44c4-954e-78e2ce5191a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.643330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3e621893-4fb5-44c4-954e-78e2ce5191a4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:34.644751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3e621893-4fb5-44c4-954e-78e2ce5191a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:34.648329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:34.662775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db89d0e0-85e8-4063-819e-58eb369bc661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:34.663511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89d0e0-85e8-4063-819e-58eb369bc661 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.663800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89d0e0-85e8-4063-819e-58eb369bc661 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.664085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db89d0e0-85e8-4063-819e-58eb369bc661 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.665527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89d0e0-85e8-4063-819e-58eb369bc661 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:34.668082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db89d0e0-85e8-4063-819e-58eb369bc661 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:34.670360 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 124/247] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:34 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:34.671820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c5fcd-efb0-46f0-8707-d7f620295bf6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.672108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c5fcd-efb0-46f0-8707-d7f620295bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.672385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0c5fcd-efb0-46f0-8707-d7f620295bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.692554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8d0c5fcd-efb0-46f0-8707-d7f620295bf6 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:34.693875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d0c5fcd-efb0-46f0-8707-d7f620295bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:34.731933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 124/248] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:34.732427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:34.734463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e621893-4fb5-44c4-954e-78e2ce5191a4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:34.773234 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 125/249] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:34.784773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c5fcd-efb0-46f0-8707-d7f620295bf6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:34.887300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e621893-4fb5-44c4-954e-78e2ce5191a4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:34.920874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0c5fcd-efb0-46f0-8707-d7f620295bf6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:34.934953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867afb5-d4b4-472d-b052-ad158a6f7d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:34.935607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867afb5-d4b4-472d-b052-ad158a6f7d8e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.935896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867afb5-d4b4-472d-b052-ad158a6f7d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.936172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867afb5-d4b4-472d-b052-ad158a6f7d8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.936976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867afb5-d4b4-472d-b052-ad158a6f7d8e None admin] RBAC: Adding query filter params (role.id=511279b616c245e2baad4d3a59cd477e, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f, user.id=1464bb3a65394de0914b2e5332b7af0b) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:34.937287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867afb5-d4b4-472d-b052-ad158a6f7d8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:34.938797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867afb5-d4b4-472d-b052-ad158a6f7d8e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:34.945054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 125/250] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:34 2022] GET /identity/v3/role_assignments?user.id=1464bb3a65394de0914b2e5332b7af0b&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=511279b616c245e2baad4d3a59cd477e => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:34.945699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867afb5-d4b4-472d-b052-ad158a6f7d8e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:34.949970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:34.968423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c4c3f-e9d3-437a-9363-d9d979b059ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:34.969108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c4c3f-e9d3-437a-9363-d9d979b059ab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:34.969512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c4c3f-e9d3-437a-9363-d9d979b059ab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:34.969838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c4c3f-e9d3-437a-9363-d9d979b059ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:34.983843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c4c3f-e9d3-437a-9363-d9d979b059ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3e2c4c3f-e9d3-437a-9363-d9d979b059ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:34.989401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:34.990407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:34.991353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 126/251] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:11:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:34.997007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c4c3f-e9d3-437a-9363-d9d979b059ab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:35.038070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:35.105326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867afb5-d4b4-472d-b052-ad158a6f7d8e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:35.141013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c4c3f-e9d3-437a-9363-d9d979b059ab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:35.157952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202c418-bdbe-4c8c-ad7e-13d13547f695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:35.158706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202c418-bdbe-4c8c-ad7e-13d13547f695 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:35.158992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202c418-bdbe-4c8c-ad7e-13d13547f695 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:35.159329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202c418-bdbe-4c8c-ad7e-13d13547f695 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:35.186018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9202c418-bdbe-4c8c-ad7e-13d13547f695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9202c418-bdbe-4c8c-ad7e-13d13547f695 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:35.191801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:35.192682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:35.193468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:35.198975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 126/252] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:35.199994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202c418-bdbe-4c8c-ad7e-13d13547f695 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:35.211401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40293f-74b6-4b7e-8bae-9f4ef2deba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:35.212053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40293f-74b6-4b7e-8bae-9f4ef2deba82 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:35.212340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40293f-74b6-4b7e-8bae-9f4ef2deba82 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:35.212616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd40293f-74b6-4b7e-8bae-9f4ef2deba82 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:35.213750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40293f-74b6-4b7e-8bae-9f4ef2deba82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:35.214077 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40293f-74b6-4b7e-8bae-9f4ef2deba82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:35.215479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd40293f-74b6-4b7e-8bae-9f4ef2deba82 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:35.225903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 127/253] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:35.232273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40293f-74b6-4b7e-8bae-9f4ef2deba82 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:35.287217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:35.365664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202c418-bdbe-4c8c-ad7e-13d13547f695 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:35.398714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:35.420450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2e40f-9c45-4020-9a45-91988e7122fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:35.421078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd40293f-74b6-4b7e-8bae-9f4ef2deba82 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:35.421711 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2e40f-9c45-4020-9a45-91988e7122fa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:35.422257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2e40f-9c45-4020-9a45-91988e7122fa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:35.422719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2e40f-9c45-4020-9a45-91988e7122fa None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:35.423658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2e40f-9c45-4020-9a45-91988e7122fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:35.424157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2e40f-9c45-4020-9a45-91988e7122fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:35.425868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2e40f-9c45-4020-9a45-91988e7122fa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:35.435781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 127/254] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:35.443051 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2e40f-9c45-4020-9a45-91988e7122fa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:35.527893 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea4678-cc5b-436f-9d30-b5a9f02a5022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:35.528658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea4678-cc5b-436f-9d30-b5a9f02a5022 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:35.528861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea4678-cc5b-436f-9d30-b5a9f02a5022 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:35.529139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea4678-cc5b-436f-9d30-b5a9f02a5022 None admin] PATH_INFO: `/v3/users/99464bba28ad437f8750d6fbc46b02ae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:35.534703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea4678-cc5b-436f-9d30-b5a9f02a5022 None admin] RBAC: Authorizing `identity:get_user(user_id=99464bba28ad437f8750d6fbc46b02ae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:35.536211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdea4678-cc5b-436f-9d30-b5a9f02a5022 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:35.538383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 128/255] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:35 2022] GET /identity/v3/users/99464bba28ad437f8750d6fbc46b02ae => generated 283 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:35.539592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedda97-3656-4e7c-8d1b-15b974632041 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:35.539882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedda97-3656-4e7c-8d1b-15b974632041 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:35.540166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbedda97-3656-4e7c-8d1b-15b974632041 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:35.541784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 129/256] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:35.546581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedda97-3656-4e7c-8d1b-15b974632041 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:35.558878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2e40f-9c45-4020-9a45-91988e7122fa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:35.587600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:35.597898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16a30e-7ded-44cd-bf38-bb87890a3cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:35.598648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16a30e-7ded-44cd-bf38-bb87890a3cdd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:35.599020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16a30e-7ded-44cd-bf38-bb87890a3cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:35.599341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16a30e-7ded-44cd-bf38-bb87890a3cdd None admin] PATH_INFO: `/v3/users/146fa5e1195c42e3b43efa0f2fde1d65` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:35.600585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16a30e-7ded-44cd-bf38-bb87890a3cdd None admin] RBAC: Authorizing `identity:get_user(user_id=146fa5e1195c42e3b43efa0f2fde1d65)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:35.602311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16a30e-7ded-44cd-bf38-bb87890a3cdd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:35.604342 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 128/257] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:35 2022] GET /identity/v3/users/146fa5e1195c42e3b43efa0f2fde1d65 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:35.605617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083726c-1796-4fce-a563-6673ef08e961 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:35.605978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083726c-1796-4fce-a563-6673ef08e961 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:35.606259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083726c-1796-4fce-a563-6673ef08e961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:35.626377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b083726c-1796-4fce-a563-6673ef08e961 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:35.627739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b083726c-1796-4fce-a563-6673ef08e961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:35.700182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 129/258] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:35.700948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083726c-1796-4fce-a563-6673ef08e961 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:35.776592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbedda97-3656-4e7c-8d1b-15b974632041 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:35.808215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083726c-1796-4fce-a563-6673ef08e961 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:35.835291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:35.843977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ddb05-df38-47f3-8f92-bdea56edc11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:35.844627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ddb05-df38-47f3-8f92-bdea56edc11e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:35.844909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ddb05-df38-47f3-8f92-bdea56edc11e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:35.845179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59ddb05-df38-47f3-8f92-bdea56edc11e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:35.848553 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420d9b2-954a-4c09-9817-e5f5084ba37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:35.849491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420d9b2-954a-4c09-9817-e5f5084ba37a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:35.850038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420d9b2-954a-4c09-9817-e5f5084ba37a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:35.850506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8420d9b2-954a-4c09-9817-e5f5084ba37a None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:35.852012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420d9b2-954a-4c09-9817-e5f5084ba37a None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:35.853764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8420d9b2-954a-4c09-9817-e5f5084ba37a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:35.855845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 130/259] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:35 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:35.856978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420d9b2-954a-4c09-9817-e5f5084ba37a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:35.898152 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:35.962990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8420d9b2-954a-4c09-9817-e5f5084ba37a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:35.984549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ddb05-df38-47f3-8f92-bdea56edc11e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:35.986115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59ddb05-df38-47f3-8f92-bdea56edc11e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:35.999849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bcd3d-a144-499f-9aaa-91c95d0b469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:36.000874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bcd3d-a144-499f-9aaa-91c95d0b469c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:36.001344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bcd3d-a144-499f-9aaa-91c95d0b469c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:36.001855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54bcd3d-a144-499f-9aaa-91c95d0b469c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:36.013414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54bcd3d-a144-499f-9aaa-91c95d0b469c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e54bcd3d-a144-499f-9aaa-91c95d0b469c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:36.018942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:36.019837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:36.020653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:36.023065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 131/260] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:36.024055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bcd3d-a144-499f-9aaa-91c95d0b469c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:36.083372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:36.132258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54bcd3d-a144-499f-9aaa-91c95d0b469c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:36.169728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4cc28-e655-471a-8271-fd935267b968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:36.170806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4cc28-e655-471a-8271-fd935267b968 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:36.171517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4cc28-e655-471a-8271-fd935267b968 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:36.172015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d4cc28-e655-471a-8271-fd935267b968 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/users/146fa5e1195c42e3b43efa0f2fde1d65/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:36.174360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4cc28-e655-471a-8271-fd935267b968 None admin] RBAC: Authorizing `identity:create_grant(project_id=e07b3a0db08041fc87412eeb161bcf6f, user_id=146fa5e1195c42e3b43efa0f2fde1d65, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:36.176580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d4cc28-e655-471a-8271-fd935267b968 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:36.198739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 132/261] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:11:36 2022] PUT /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/users/146fa5e1195c42e3b43efa0f2fde1d65/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:36.201295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4cc28-e655-471a-8271-fd935267b968 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:36.280256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 130/262] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 734 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:36.286056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ddb05-df38-47f3-8f92-bdea56edc11e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:36.416406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:36.424359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d4cc28-e655-471a-8271-fd935267b968 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:36.468175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:36.499887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59ddb05-df38-47f3-8f92-bdea56edc11e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:36.522241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5a53c-a8fb-4cf8-9959-3d14b1791dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:36.522947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a53c-a8fb-4cf8-9959-3d14b1791dde None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:36.523255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a53c-a8fb-4cf8-9959-3d14b1791dde None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:36.523711 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a53c-a8fb-4cf8-9959-3d14b1791dde None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:36.524889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5a53c-a8fb-4cf8-9959-3d14b1791dde None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:36.525197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5a53c-a8fb-4cf8-9959-3d14b1791dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:36.528014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5a53c-a8fb-4cf8-9959-3d14b1791dde None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:36.537777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 133/263] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:36.544129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5a53c-a8fb-4cf8-9959-3d14b1791dde None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:36.559717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be5ce5-e882-41eb-a8a9-2c3c841860be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:36.560391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be5ce5-e882-41eb-a8a9-2c3c841860be None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:36.560668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be5ce5-e882-41eb-a8a9-2c3c841860be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:36.560938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be5ce5-e882-41eb-a8a9-2c3c841860be None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:36.564836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be5ce5-e882-41eb-a8a9-2c3c841860be None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:36.566459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0be5ce5-e882-41eb-a8a9-2c3c841860be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:36.568622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 131/264] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:36 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:36.598200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be5ce5-e882-41eb-a8a9-2c3c841860be None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:36.671522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5a53c-a8fb-4cf8-9959-3d14b1791dde None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:36.717505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46912a1-2f82-4cdc-b032-6c6a178668d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:36.718482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46912a1-2f82-4cdc-b032-6c6a178668d9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:36.718915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46912a1-2f82-4cdc-b032-6c6a178668d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:36.719336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46912a1-2f82-4cdc-b032-6c6a178668d9 None admin] PATH_INFO: `/v3/users/99464bba28ad437f8750d6fbc46b02ae` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:36.720744 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46912a1-2f82-4cdc-b032-6c6a178668d9 None admin] RBAC: Authorizing `identity:get_user(user_id=99464bba28ad437f8750d6fbc46b02ae)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:36.722501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46912a1-2f82-4cdc-b032-6c6a178668d9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:36.724648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 134/265] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:36 2022] GET /identity/v3/users/99464bba28ad437f8750d6fbc46b02ae => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:36.734811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46912a1-2f82-4cdc-b032-6c6a178668d9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:36.806137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:36.839807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be5ce5-e882-41eb-a8a9-2c3c841860be None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:36.893668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46912a1-2f82-4cdc-b032-6c6a178668d9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:36.953260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bab8e3-a68d-42f4-a353-261e8a46294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:36.953896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:36.954080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab8e3-a68d-42f4-a353-261e8a46294a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:36.954323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab8e3-a68d-42f4-a353-261e8a46294a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:36.954594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bab8e3-a68d-42f4-a353-261e8a46294a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:36.981153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408097e3-dfad-4804-b8c6-0789b398df60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:36.981833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408097e3-dfad-4804-b8c6-0789b398df60 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:36.982119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408097e3-dfad-4804-b8c6-0789b398df60 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:36.982395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408097e3-dfad-4804-b8c6-0789b398df60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:36.983561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408097e3-dfad-4804-b8c6-0789b398df60 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f, user.id=99464bba28ad437f8750d6fbc46b02ae) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:36.983878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408097e3-dfad-4804-b8c6-0789b398df60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:36.985305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408097e3-dfad-4804-b8c6-0789b398df60 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:37.008148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 132/266] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:36 2022] GET /identity/v3/role_assignments?user.id=99464bba28ad437f8750d6fbc46b02ae&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 608 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:37.105966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bab8e3-a68d-42f4-a353-261e8a46294a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:37.107633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bab8e3-a68d-42f4-a353-261e8a46294a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:37.174460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:37.243948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 135/267] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:37.249496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408097e3-dfad-4804-b8c6-0789b398df60 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:37.404479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d915efd-00f9-448e-8901-de519a9f1794 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:37.404819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d915efd-00f9-448e-8901-de519a9f1794 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:37.405029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d915efd-00f9-448e-8901-de519a9f1794 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:37.406559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 136/268] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:11:37.408731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408097e3-dfad-4804-b8c6-0789b398df60 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:37.415191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090bed2-938b-4d5c-9d41-e870dc302fa3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:37.415483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090bed2-938b-4d5c-9d41-e870dc302fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:37.415730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5090bed2-938b-4d5c-9d41-e870dc302fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:37.437848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5090bed2-938b-4d5c-9d41-e870dc302fa3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:37.439858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5090bed2-938b-4d5c-9d41-e870dc302fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:37.462724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eedf0-905f-4dca-8a08-386e78607cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:37.463701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eedf0-905f-4dca-8a08-386e78607cda None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:37.464177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eedf0-905f-4dca-8a08-386e78607cda None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:37.464613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5eedf0-905f-4dca-8a08-386e78607cda None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:37.466103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eedf0-905f-4dca-8a08-386e78607cda None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:37.468214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5eedf0-905f-4dca-8a08-386e78607cda None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:37.470549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 133/269] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:37 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:37.477171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eedf0-905f-4dca-8a08-386e78607cda None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:37.573602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 137/270] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:37.604920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:37.608761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090bed2-938b-4d5c-9d41-e870dc302fa3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:37.657517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5eedf0-905f-4dca-8a08-386e78607cda None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:37.703719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d0e10-8012-4b59-a824-bab3d149ea9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:37.704761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d0e10-8012-4b59-a824-bab3d149ea9c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:37.705233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d0e10-8012-4b59-a824-bab3d149ea9c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:37.705707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d0e10-8012-4b59-a824-bab3d149ea9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:37.706926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d0e10-8012-4b59-a824-bab3d149ea9c None admin] RBAC: Adding query filter params (role.id=5e0c08646046426385776839f1f168cd, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f, user.id=99464bba28ad437f8750d6fbc46b02ae) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:37.707458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d0e10-8012-4b59-a824-bab3d149ea9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:37.707910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:37.710475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d0e10-8012-4b59-a824-bab3d149ea9c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:37.718916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 134/271] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:37 2022] GET /identity/v3/role_assignments?user.id=99464bba28ad437f8750d6fbc46b02ae&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=5e0c08646046426385776839f1f168cd => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:37.774293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5090bed2-938b-4d5c-9d41-e870dc302fa3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:37.838752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616572f-5498-43e5-be81-220ed76957dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:37.839363 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616572f-5498-43e5-be81-220ed76957dc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:37.839652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616572f-5498-43e5-be81-220ed76957dc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:37.839922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e616572f-5498-43e5-be81-220ed76957dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:37.855115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13557168-e5ca-4dd5-9c10-888cc5806965 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:37.855460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13557168-e5ca-4dd5-9c10-888cc5806965 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:37.855680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13557168-e5ca-4dd5-9c10-888cc5806965 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:37.857349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 135/272] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:37.860812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e616572f-5498-43e5-be81-220ed76957dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:37.865845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76f261-e860-4a72-b600-b091469a4969 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:37.866126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76f261-e860-4a72-b600-b091469a4969 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:37.866406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b76f261-e860-4a72-b600-b091469a4969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e616572f-5498-43e5-be81-220ed76957dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:37.867329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:37.868165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:37.868945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:37.871355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 138/273] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:37.878446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616572f-5498-43e5-be81-220ed76957dc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:37.904122 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1b76f261-e860-4a72-b600-b091469a4969 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:37.905501 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1b76f261-e860-4a72-b600-b091469a4969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:38.006542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 136/274] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:38.007855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3681d-d864-4657-b594-b2252212d749 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.008147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3681d-d864-4657-b594-b2252212d749 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.008428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb3681d-d864-4657-b594-b2252212d749 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.014148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 137/275] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:38.023240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60a112-b2de-4b18-85e6-af3fff6d4e30 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.023528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60a112-b2de-4b18-85e6-af3fff6d4e30 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.023808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e60a112-b2de-4b18-85e6-af3fff6d4e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.034662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e616572f-5498-43e5-be81-220ed76957dc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:38.048470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1e60a112-b2de-4b18-85e6-af3fff6d4e30 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:38.049700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1e60a112-b2de-4b18-85e6-af3fff6d4e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:38.071463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:38.078453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675a75c-9a90-48bf-aa12-5c9c52bba459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:38.079470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675a75c-9a90-48bf-aa12-5c9c52bba459 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.079765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675a75c-9a90-48bf-aa12-5c9c52bba459 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.080042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675a75c-9a90-48bf-aa12-5c9c52bba459 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.080786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675a75c-9a90-48bf-aa12-5c9c52bba459 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:38.081137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675a75c-9a90-48bf-aa12-5c9c52bba459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:38.082661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675a75c-9a90-48bf-aa12-5c9c52bba459 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:38.092183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 139/276] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:38.093094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675a75c-9a90-48bf-aa12-5c9c52bba459 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:38.115242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:38.141817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 138/277] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:38.143012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60a112-b2de-4b18-85e6-af3fff6d4e30 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:38.216334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675a75c-9a90-48bf-aa12-5c9c52bba459 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:38.255277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a689e-818b-4575-8266-86a7caf6c3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:38.256144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a689e-818b-4575-8266-86a7caf6c3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.256521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a689e-818b-4575-8266-86a7caf6c3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.256871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a689e-818b-4575-8266-86a7caf6c3e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.265511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e60a112-b2de-4b18-85e6-af3fff6d4e30 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:38.270040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a689e-818b-4575-8266-86a7caf6c3e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d55a689e-818b-4575-8266-86a7caf6c3e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:38.276472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:38.277444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:38.278698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 140/278] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:38.281649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a689e-818b-4575-8266-86a7caf6c3e3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:38.313616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971c816-1ea8-485e-a071-7969550061dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:38.314463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971c816-1ea8-485e-a071-7969550061dc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.314750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971c816-1ea8-485e-a071-7969550061dc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.315253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971c816-1ea8-485e-a071-7969550061dc None admin] PATH_INFO: `/v3/users/27a92ada0271441b8a7ceefdc2625bc3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.330519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971c816-1ea8-485e-a071-7969550061dc None admin] RBAC: Authorizing `identity:get_user(user_id=27a92ada0271441b8a7ceefdc2625bc3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:38.332120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f971c816-1ea8-485e-a071-7969550061dc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:38.333963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 139/279] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:38 2022] GET /identity/v3/users/27a92ada0271441b8a7ceefdc2625bc3 => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:38.334784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971c816-1ea8-485e-a071-7969550061dc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:38.352765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:38.383513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:38.402284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a689e-818b-4575-8266-86a7caf6c3e3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:38.440850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890381e0-b03e-4fb4-a40c-0dd6a88250ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:38.441529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890381e0-b03e-4fb4-a40c-0dd6a88250ee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.442127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890381e0-b03e-4fb4-a40c-0dd6a88250ee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.442409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890381e0-b03e-4fb4-a40c-0dd6a88250ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.443897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890381e0-b03e-4fb4-a40c-0dd6a88250ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:38.446044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890381e0-b03e-4fb4-a40c-0dd6a88250ee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:38.453430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 141/280] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:38.453430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890381e0-b03e-4fb4-a40c-0dd6a88250ee None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:38.453690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f971c816-1ea8-485e-a071-7969550061dc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:38.490072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88ea12-1ee0-4a03-b4ba-85e8cadbdf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:38.490703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88ea12-1ee0-4a03-b4ba-85e8cadbdf12 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.490972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88ea12-1ee0-4a03-b4ba-85e8cadbdf12 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.491244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88ea12-1ee0-4a03-b4ba-85e8cadbdf12 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.491967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88ea12-1ee0-4a03-b4ba-85e8cadbdf12 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:38.492268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88ea12-1ee0-4a03-b4ba-85e8cadbdf12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:38.493628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d88ea12-1ee0-4a03-b4ba-85e8cadbdf12 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:38.502699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 140/281] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:38.503487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88ea12-1ee0-4a03-b4ba-85e8cadbdf12 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:38.568605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890381e0-b03e-4fb4-a40c-0dd6a88250ee None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:38.593087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:38.606393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918268b5-3678-4d1c-b2e8-8cc28335da98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:38.607236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918268b5-3678-4d1c-b2e8-8cc28335da98 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.607501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918268b5-3678-4d1c-b2e8-8cc28335da98 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.607771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918268b5-3678-4d1c-b2e8-8cc28335da98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.608949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918268b5-3678-4d1c-b2e8-8cc28335da98 None admin] RBAC: Adding query filter params (role.id=64dfc66a8abe4468ac4f1ca071d28bd4, user.id=27a92ada0271441b8a7ceefdc2625bc3, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:38.609234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918268b5-3678-4d1c-b2e8-8cc28335da98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:38.610800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918268b5-3678-4d1c-b2e8-8cc28335da98 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:38.613052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88ea12-1ee0-4a03-b4ba-85e8cadbdf12 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:38.615023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:38.617184 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 142/282] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Jul 26 07:11:38 2022] GET /identity/v3/role_assignments?user.id=27a92ada0271441b8a7ceefdc2625bc3&scope.system=all&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:38.617633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918268b5-3678-4d1c-b2e8-8cc28335da98 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:38.661112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfb4e0-7c2b-4de6-a13a-558ac6af7cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:38.662223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfb4e0-7c2b-4de6-a13a-558ac6af7cba None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.662752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfb4e0-7c2b-4de6-a13a-558ac6af7cba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.663219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfb4e0-7c2b-4de6-a13a-558ac6af7cba None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.664131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfb4e0-7c2b-4de6-a13a-558ac6af7cba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:38.665953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfb4e0-7c2b-4de6-a13a-558ac6af7cba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:38.694982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 141/283] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:11:38 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:11:38.757440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918268b5-3678-4d1c-b2e8-8cc28335da98 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:38.806878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99505c6e-f6e4-4ae6-8fcc-7e030f402c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:38.807569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99505c6e-f6e4-4ae6-8fcc-7e030f402c90 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.807846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99505c6e-f6e4-4ae6-8fcc-7e030f402c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.808117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99505c6e-f6e4-4ae6-8fcc-7e030f402c90 None admin] PATH_INFO: `/v3/users/1464bb3a65394de0914b2e5332b7af0b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.809453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99505c6e-f6e4-4ae6-8fcc-7e030f402c90 None admin] RBAC: Authorizing `identity:get_user(user_id=1464bb3a65394de0914b2e5332b7af0b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:38.811015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99505c6e-f6e4-4ae6-8fcc-7e030f402c90 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:38.813311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 143/284] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:38 2022] GET /identity/v3/users/1464bb3a65394de0914b2e5332b7af0b => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:38.819500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfb4e0-7c2b-4de6-a13a-558ac6af7cba None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:38.943419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfb4e0-7c2b-4de6-a13a-558ac6af7cba None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:38.981695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841955b-57ff-4bf4-bcdf-c14615a81990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:38.982392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841955b-57ff-4bf4-bcdf-c14615a81990 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:38.982669 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841955b-57ff-4bf4-bcdf-c14615a81990 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:38.982942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841955b-57ff-4bf4-bcdf-c14615a81990 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:38.984552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841955b-57ff-4bf4-bcdf-c14615a81990 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:38.986115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841955b-57ff-4bf4-bcdf-c14615a81990 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:38.988191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 142/285] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:38 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:38.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99505c6e-f6e4-4ae6-8fcc-7e030f402c90 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:39.046813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71942e09-9b74-4c1c-9956-0e34c1437a37 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:39.047104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71942e09-9b74-4c1c-9956-0e34c1437a37 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:39.047389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71942e09-9b74-4c1c-9956-0e34c1437a37 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:39.049336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 143/286] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:39.054568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef63fb-84ae-4989-89df-d8d6b2158a4d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:39.054782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef63fb-84ae-4989-89df-d8d6b2158a4d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:39.055391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef63fb-84ae-4989-89df-d8d6b2158a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:39.075148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:39.076909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-65ef63fb-84ae-4989-89df-d8d6b2158a4d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:39.079142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65ef63fb-84ae-4989-89df-d8d6b2158a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:39.109268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:39.147377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99505c6e-f6e4-4ae6-8fcc-7e030f402c90 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:39.185308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 144/287] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:39.208074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef63fb-84ae-4989-89df-d8d6b2158a4d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:39.216510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154a0ef-066a-49e8-a551-f22036dd1efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:39.217553 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154a0ef-066a-49e8-a551-f22036dd1efb None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:39.218124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154a0ef-066a-49e8-a551-f22036dd1efb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:39.218594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a154a0ef-066a-49e8-a551-f22036dd1efb None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/users/1464bb3a65394de0914b2e5332b7af0b/roles/511279b616c245e2baad4d3a59cd477e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:39.220899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154a0ef-066a-49e8-a551-f22036dd1efb None admin] RBAC: Authorizing `identity:create_grant(project_id=e07b3a0db08041fc87412eeb161bcf6f, user_id=1464bb3a65394de0914b2e5332b7af0b, role_id=511279b616c245e2baad4d3a59cd477e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:39.222702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a154a0ef-066a-49e8-a551-f22036dd1efb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:39.240012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 144/288] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:11:38 2022] PUT /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/users/1464bb3a65394de0914b2e5332b7af0b/roles/511279b616c245e2baad4d3a59cd477e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:39.343814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef63fb-84ae-4989-89df-d8d6b2158a4d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:39.384301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245043f1-b33a-45df-9227-ed4025f6cf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:39.385446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245043f1-b33a-45df-9227-ed4025f6cf8e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:39.385979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245043f1-b33a-45df-9227-ed4025f6cf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:39.386460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245043f1-b33a-45df-9227-ed4025f6cf8e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:39.398778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245043f1-b33a-45df-9227-ed4025f6cf8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-245043f1-b33a-45df-9227-ed4025f6cf8e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:39.404371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:39.405335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:39.406235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:39.409185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 145/289] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:39.415750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154a0ef-066a-49e8-a551-f22036dd1efb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:39.534348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a154a0ef-066a-49e8-a551-f22036dd1efb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:39.551436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:39.574924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad05fc-198b-4e7e-b640-2e46c5f4d414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:39.575576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad05fc-198b-4e7e-b640-2e46c5f4d414 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:39.575865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad05fc-198b-4e7e-b640-2e46c5f4d414 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:39.576147 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad05fc-198b-4e7e-b640-2e46c5f4d414 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:39.576936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad05fc-198b-4e7e-b640-2e46c5f4d414 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:39.577212 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad05fc-198b-4e7e-b640-2e46c5f4d414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:39.578848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad05fc-198b-4e7e-b640-2e46c5f4d414 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:39.589303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 145/290] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:39.595045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245043f1-b33a-45df-9227-ed4025f6cf8e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:39.618014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:39.748642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245043f1-b33a-45df-9227-ed4025f6cf8e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:39.808117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02778dd6-6d36-4f57-85cc-0e5ec1a3a1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:39.809154 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02778dd6-6d36-4f57-85cc-0e5ec1a3a1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:39.809632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02778dd6-6d36-4f57-85cc-0e5ec1a3a1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:39.810133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02778dd6-6d36-4f57-85cc-0e5ec1a3a1d3 None admin] PATH_INFO: `/v3/users/146fa5e1195c42e3b43efa0f2fde1d65` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:39.811614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02778dd6-6d36-4f57-85cc-0e5ec1a3a1d3 None admin] RBAC: Authorizing `identity:get_user(user_id=146fa5e1195c42e3b43efa0f2fde1d65)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:39.813269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02778dd6-6d36-4f57-85cc-0e5ec1a3a1d3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:39.815210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 146/291] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:39 2022] GET /identity/v3/users/146fa5e1195c42e3b43efa0f2fde1d65 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:39.821182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad05fc-198b-4e7e-b640-2e46c5f4d414 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:39.935964 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad05fc-198b-4e7e-b640-2e46c5f4d414 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:39.972954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc594c-b323-4b94-bdbb-5e1a5a0ff7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:39.973596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc594c-b323-4b94-bdbb-5e1a5a0ff7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:39.973907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc594c-b323-4b94-bdbb-5e1a5a0ff7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:39.974182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcc594c-b323-4b94-bdbb-5e1a5a0ff7c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:40.024772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:40.049246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc594c-b323-4b94-bdbb-5e1a5a0ff7c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:40.050929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcc594c-b323-4b94-bdbb-5e1a5a0ff7c1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:40.154696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 146/292] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:40.160543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02778dd6-6d36-4f57-85cc-0e5ec1a3a1d3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:40.271607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:40.293426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02778dd6-6d36-4f57-85cc-0e5ec1a3a1d3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:40.336923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b89e17d-367e-4237-8b29-172b8ce93523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:40.337584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89e17d-367e-4237-8b29-172b8ce93523 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:40.337893 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89e17d-367e-4237-8b29-172b8ce93523 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:40.338167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b89e17d-367e-4237-8b29-172b8ce93523 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:40.339339 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89e17d-367e-4237-8b29-172b8ce93523 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:40.340796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b89e17d-367e-4237-8b29-172b8ce93523 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:40.342834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 147/293] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:40 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:40.348957 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc594c-b323-4b94-bdbb-5e1a5a0ff7c1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:40.427670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:40.451815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcc594c-b323-4b94-bdbb-5e1a5a0ff7c1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:40.491745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abcea58-6b64-483b-8d0d-d6afe9b18b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:40.492439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcea58-6b64-483b-8d0d-d6afe9b18b45 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:40.492720 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcea58-6b64-483b-8d0d-d6afe9b18b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:40.492994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abcea58-6b64-483b-8d0d-d6afe9b18b45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:40.493879 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcea58-6b64-483b-8d0d-d6afe9b18b45 None admin] RBAC: Adding query filter params (role.id=64dfc66a8abe4468ac4f1ca071d28bd4, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f, user.id=146fa5e1195c42e3b43efa0f2fde1d65) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:40.494191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcea58-6b64-483b-8d0d-d6afe9b18b45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:40.495636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abcea58-6b64-483b-8d0d-d6afe9b18b45 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:40.502191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 147/294] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:40 2022] GET /identity/v3/role_assignments?user.id=146fa5e1195c42e3b43efa0f2fde1d65&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:41.111889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca04cef-0304-4544-961a-a194122a1bed None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.112470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca04cef-0304-4544-961a-a194122a1bed None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.112470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca04cef-0304-4544-961a-a194122a1bed None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.114533 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 148/295] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:41.120097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c0515-c234-4334-a1f8-f85b1cd1fc24 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.120554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c0515-c234-4334-a1f8-f85b1cd1fc24 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.120653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c0515-c234-4334-a1f8-f85b1cd1fc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.140814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-614c0515-c234-4334-a1f8-f85b1cd1fc24 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:41.142050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-614c0515-c234-4334-a1f8-f85b1cd1fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:41.163701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e835c0f-e0d1-4db3-b432-ee30100dcf01 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.164319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e835c0f-e0d1-4db3-b432-ee30100dcf01 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.164944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e835c0f-e0d1-4db3-b432-ee30100dcf01 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.167255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 149/296] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:41.173681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6f21f-3b6c-406a-bfaf-9137f1592610 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.174293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6f21f-3b6c-406a-bfaf-9137f1592610 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.176256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6f21f-3b6c-406a-bfaf-9137f1592610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.195300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9bc6f21f-3b6c-406a-bfaf-9137f1592610 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:41.196827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9bc6f21f-3b6c-406a-bfaf-9137f1592610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:41.239512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 148/297] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:41.261433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c0515-c234-4334-a1f8-f85b1cd1fc24 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:41.277760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 150/298] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:41.294212 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6f21f-3b6c-406a-bfaf-9137f1592610 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:41.354718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:41.383951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:41.405216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc6f21f-3b6c-406a-bfaf-9137f1592610 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:41.405410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c0515-c234-4334-a1f8-f85b1cd1fc24 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:41.442675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa487953-0ed1-4e9d-99c2-c66e66499970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:41.443737 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa487953-0ed1-4e9d-99c2-c66e66499970 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.444213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa487953-0ed1-4e9d-99c2-c66e66499970 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.444673 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa487953-0ed1-4e9d-99c2-c66e66499970 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.451044 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552b8e6-3e87-49d9-9934-0f3ef7215a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:41.452092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b8e6-3e87-49d9-9934-0f3ef7215a9a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.452626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b8e6-3e87-49d9-9934-0f3ef7215a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.453045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552b8e6-3e87-49d9-9934-0f3ef7215a9a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.458794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa487953-0ed1-4e9d-99c2-c66e66499970 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fa487953-0ed1-4e9d-99c2-c66e66499970 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:41.464307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:41.465389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:41.466462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:41.468125 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 149/299] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:41.468572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552b8e6-3e87-49d9-9934-0f3ef7215a9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d552b8e6-3e87-49d9-9934-0f3ef7215a9a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:41.473980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:41.475073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:41.476066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:41.476888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa487953-0ed1-4e9d-99c2-c66e66499970 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:41.478757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 151/300] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:41.486494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552b8e6-3e87-49d9-9934-0f3ef7215a9a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:41.593357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa487953-0ed1-4e9d-99c2-c66e66499970 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:41.596609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552b8e6-3e87-49d9-9934-0f3ef7215a9a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:41.623851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:41.630980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51dac9-de3d-4c3a-a958-d6385d2931da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:41.631619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51dac9-de3d-4c3a-a958-d6385d2931da None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.631899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51dac9-de3d-4c3a-a958-d6385d2931da None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.632163 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51dac9-de3d-4c3a-a958-d6385d2931da None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.632891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51dac9-de3d-4c3a-a958-d6385d2931da None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:41.633195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51dac9-de3d-4c3a-a958-d6385d2931da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:41.633288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4508b49-0761-48ec-9499-6b3b6f0bebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:41.633932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4508b49-0761-48ec-9499-6b3b6f0bebf9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.634269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4508b49-0761-48ec-9499-6b3b6f0bebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.634496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4508b49-0761-48ec-9499-6b3b6f0bebf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.634662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51dac9-de3d-4c3a-a958-d6385d2931da None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:41.635221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4508b49-0761-48ec-9499-6b3b6f0bebf9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:41.635523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4508b49-0761-48ec-9499-6b3b6f0bebf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:41.637412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4508b49-0761-48ec-9499-6b3b6f0bebf9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:41.643847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 150/301] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:41.645991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a27d8-50b1-4242-a6fb-764f61a91873 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.646424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a27d8-50b1-4242-a6fb-764f61a91873 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.646849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a27d8-50b1-4242-a6fb-764f61a91873 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.648102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 152/302] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:41.648436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 151/303] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:41.653417 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4508b49-0761-48ec-9499-6b3b6f0bebf9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:41.654138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654325af-4fca-4a5a-89ff-95b094bee143 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.654445 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654325af-4fca-4a5a-89ff-95b094bee143 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.654716 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654325af-4fca-4a5a-89ff-95b094bee143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.657433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:41.673854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-654325af-4fca-4a5a-89ff-95b094bee143 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:41.675144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-654325af-4fca-4a5a-89ff-95b094bee143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:41.749180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 152/304] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:41.750405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654325af-4fca-4a5a-89ff-95b094bee143 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:41.760233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4508b49-0761-48ec-9499-6b3b6f0bebf9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:41.801050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc8640-3bdd-493b-b397-4b9795fd7b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:41.801715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc8640-3bdd-493b-b397-4b9795fd7b2b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.802040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc8640-3bdd-493b-b397-4b9795fd7b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.802316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc8640-3bdd-493b-b397-4b9795fd7b2b None admin] PATH_INFO: `/v3/users/27a92ada0271441b8a7ceefdc2625bc3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.803538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc8640-3bdd-493b-b397-4b9795fd7b2b None admin] RBAC: Authorizing `identity:get_user(user_id=27a92ada0271441b8a7ceefdc2625bc3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:41.804983 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc8640-3bdd-493b-b397-4b9795fd7b2b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:41.806821 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 153/305] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:41 2022] GET /identity/v3/users/27a92ada0271441b8a7ceefdc2625bc3 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:41.807431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc8640-3bdd-493b-b397-4b9795fd7b2b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:41.872796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654325af-4fca-4a5a-89ff-95b094bee143 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:41.880329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:41.895998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:41.907669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc48f9-d5b5-4d67-9db9-d66c4fb15f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:41.908323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc48f9-d5b5-4d67-9db9-d66c4fb15f74 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.908599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc48f9-d5b5-4d67-9db9-d66c4fb15f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.908866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bc48f9-d5b5-4d67-9db9-d66c4fb15f74 None admin] PATH_INFO: `/v3/users/ff54e343564b44f29ee239d821e6b440` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.910600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc8640-3bdd-493b-b397-4b9795fd7b2b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:41.922486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc48f9-d5b5-4d67-9db9-d66c4fb15f74 None admin] RBAC: Authorizing `identity:get_user(user_id=ff54e343564b44f29ee239d821e6b440)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:41.923979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bc48f9-d5b5-4d67-9db9-d66c4fb15f74 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:41.925797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 153/306] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:41 2022] GET /identity/v3/users/ff54e343564b44f29ee239d821e6b440 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:41.926540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc48f9-d5b5-4d67-9db9-d66c4fb15f74 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:41.949931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34f71b-c4c1-4ba8-a5d4-8c3e2badae72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:41.950597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34f71b-c4c1-4ba8-a5d4-8c3e2badae72 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.950885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34f71b-c4c1-4ba8-a5d4-8c3e2badae72 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.951164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34f71b-c4c1-4ba8-a5d4-8c3e2badae72 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.952356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34f71b-c4c1-4ba8-a5d4-8c3e2badae72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:41.953839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34f71b-c4c1-4ba8-a5d4-8c3e2badae72 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:41.955452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 154/307] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:41.955452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a87ed-573e-45fd-b129-ae9832b932ae None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:41.955452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a87ed-573e-45fd-b129-ae9832b932ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:41.955452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a87ed-573e-45fd-b129-ae9832b932ae None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:41.955452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 155/308] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:41.955452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a87ed-573e-45fd-b129-ae9832b932ae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:42.028917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bc48f9-d5b5-4d67-9db9-d66c4fb15f74 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:42.061276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a87ed-573e-45fd-b129-ae9832b932ae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:42.062720 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79724c69-8b7a-4fce-b4f5-f2e6fef9bb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:42.063407 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79724c69-8b7a-4fce-b4f5-f2e6fef9bb55 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:42.063685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79724c69-8b7a-4fce-b4f5-f2e6fef9bb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:42.063958 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79724c69-8b7a-4fce-b4f5-f2e6fef9bb55 None admin] PATH_INFO: `/v3/system/users/27a92ada0271441b8a7ceefdc2625bc3/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:42.065505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79724c69-8b7a-4fce-b4f5-f2e6fef9bb55 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=27a92ada0271441b8a7ceefdc2625bc3, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:42.066968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79724c69-8b7a-4fce-b4f5-f2e6fef9bb55 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:42.076385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 154/309] 10.222.0.39 () {58 vars in 1334 bytes} [Tue Jul 26 07:11:41 2022] PUT /identity/v3/system/users/27a92ada0271441b8a7ceefdc2625bc3/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:42.077296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79724c69-8b7a-4fce-b4f5-f2e6fef9bb55 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:42.102353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efc6d44-eeff-47d9-99d7-0e58c90d1fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:42.103307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc6d44-eeff-47d9-99d7-0e58c90d1fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:42.103649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc6d44-eeff-47d9-99d7-0e58c90d1fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:42.103923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efc6d44-eeff-47d9-99d7-0e58c90d1fe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:42.105104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc6d44-eeff-47d9-99d7-0e58c90d1fe3 None admin] RBAC: Adding query filter params (role.id=511279b616c245e2baad4d3a59cd477e, user.id=ff54e343564b44f29ee239d821e6b440, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:42.105958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc6d44-eeff-47d9-99d7-0e58c90d1fe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:42.107458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efc6d44-eeff-47d9-99d7-0e58c90d1fe3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:42.116521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 156/310] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Jul 26 07:11:41 2022] GET /identity/v3/role_assignments?user.id=ff54e343564b44f29ee239d821e6b440&scope.system=all&role.id=511279b616c245e2baad4d3a59cd477e => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:42.117967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c00b4-f74b-43e5-819e-0059ad9566d2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:42.118256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c00b4-f74b-43e5-819e-0059ad9566d2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:42.118532 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c00b4-f74b-43e5-819e-0059ad9566d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:42.127754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:42.141268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-814c00b4-f74b-43e5-819e-0059ad9566d2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:42.142606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-814c00b4-f74b-43e5-819e-0059ad9566d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:42.158698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:42.240304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79724c69-8b7a-4fce-b4f5-f2e6fef9bb55 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:42.244483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 157/311] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:42.266596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c00b4-f74b-43e5-819e-0059ad9566d2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:42.284366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058a02d-f89a-44e2-8dba-f0d358f17d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:42.285072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058a02d-f89a-44e2-8dba-f0d358f17d5c None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:42.285358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058a02d-f89a-44e2-8dba-f0d358f17d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:42.285636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d058a02d-f89a-44e2-8dba-f0d358f17d5c None admin] PATH_INFO: `/v3/groups` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:42.286359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058a02d-f89a-44e2-8dba-f0d358f17d5c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:42.287905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d058a02d-f89a-44e2-8dba-f0d358f17d5c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:42.309015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 155/312] 10.222.0.39 () {60 vars in 1134 bytes} [Tue Jul 26 07:11:42 2022] POST /identity/v3/groups => generated 229 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:11:42.404335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814c00b4-f74b-43e5-819e-0059ad9566d2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:42.458020 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c0a16-79e1-483c-b23b-d70e805252b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:42.458703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c0a16-79e1-483c-b23b-d70e805252b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:42.459004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c0a16-79e1-483c-b23b-d70e805252b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:42.459320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169c0a16-79e1-483c-b23b-d70e805252b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:42.461520 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:42.474148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169c0a16-79e1-483c-b23b-d70e805252b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-169c0a16-79e1-483c-b23b-d70e805252b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:42.479176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:42.480052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:42.480829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 158/313] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:42.488043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058a02d-f89a-44e2-8dba-f0d358f17d5c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:42.598195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d058a02d-f89a-44e2-8dba-f0d358f17d5c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:42.603156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:42.640160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93733d85-20c9-41d2-a4fa-2e7429171158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:42.640800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93733d85-20c9-41d2-a4fa-2e7429171158 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:42.641083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93733d85-20c9-41d2-a4fa-2e7429171158 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:42.641355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93733d85-20c9-41d2-a4fa-2e7429171158 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:42.642118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93733d85-20c9-41d2-a4fa-2e7429171158 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:42.642422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93733d85-20c9-41d2-a4fa-2e7429171158 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:42.643864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93733d85-20c9-41d2-a4fa-2e7429171158 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:42.653248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 156/314] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:42.659008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c0a16-79e1-483c-b23b-d70e805252b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:42.762260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169c0a16-79e1-483c-b23b-d70e805252b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:42.802738 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffae597-770a-4d78-ae5d-ebaee4318c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:42.803403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffae597-770a-4d78-ae5d-ebaee4318c86 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:42.803687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffae597-770a-4d78-ae5d-ebaee4318c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:42.803966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffae597-770a-4d78-ae5d-ebaee4318c86 None admin] PATH_INFO: `/v3/users/1464bb3a65394de0914b2e5332b7af0b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:42.805188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffae597-770a-4d78-ae5d-ebaee4318c86 None admin] RBAC: Authorizing `identity:get_user(user_id=1464bb3a65394de0914b2e5332b7af0b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:42.806686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffae597-770a-4d78-ae5d-ebaee4318c86 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:42.808524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 159/315] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:42 2022] GET /identity/v3/users/1464bb3a65394de0914b2e5332b7af0b => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:42.814221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93733d85-20c9-41d2-a4fa-2e7429171158 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:42.921827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93733d85-20c9-41d2-a4fa-2e7429171158 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:42.957549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e997455-6007-42c9-913f-93169dde49d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:42.958225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e997455-6007-42c9-913f-93169dde49d5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:42.958513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e997455-6007-42c9-913f-93169dde49d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:42.958787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e997455-6007-42c9-913f-93169dde49d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:42.994697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:43.029673 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e997455-6007-42c9-913f-93169dde49d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:43.031192 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e997455-6007-42c9-913f-93169dde49d5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:43.131771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 157/316] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:43.137645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffae597-770a-4d78-ae5d-ebaee4318c86 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:43.200533 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:43.240356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffae597-770a-4d78-ae5d-ebaee4318c86 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:43.277399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aedd582-ab55-429b-ae08-e6856989bf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:43.278101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedd582-ab55-429b-ae08-e6856989bf2f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:43.278383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedd582-ab55-429b-ae08-e6856989bf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:43.278659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aedd582-ab55-429b-ae08-e6856989bf2f None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:43.279948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aedd582-ab55-429b-ae08-e6856989bf2f None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:43.281375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aedd582-ab55-429b-ae08-e6856989bf2f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:43.288218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 160/317] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:43 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:43.289641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e997455-6007-42c9-913f-93169dde49d5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:43.358812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:43.393782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e997455-6007-42c9-913f-93169dde49d5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:43.430559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7350c23-5dac-465b-b8f3-4262b7899a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:43.431198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350c23-5dac-465b-b8f3-4262b7899a16 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:43.431481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350c23-5dac-465b-b8f3-4262b7899a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:43.431757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7350c23-5dac-465b-b8f3-4262b7899a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:43.432575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7350c23-5dac-465b-b8f3-4262b7899a16 None admin] RBAC: Adding query filter params (role.id=511279b616c245e2baad4d3a59cd477e, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f, user.id=1464bb3a65394de0914b2e5332b7af0b) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:43.432887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7350c23-5dac-465b-b8f3-4262b7899a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:43.434373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7350c23-5dac-465b-b8f3-4262b7899a16 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:43.445867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 158/318] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:11:43 2022] GET /identity/v3/role_assignments?user.id=1464bb3a65394de0914b2e5332b7af0b&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=511279b616c245e2baad4d3a59cd477e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:44.182992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3e3d5-e5dc-4b83-ac43-57697adfab54 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.183732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3e3d5-e5dc-4b83-ac43-57697adfab54 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.183732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3e3d5-e5dc-4b83-ac43-57697adfab54 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.184950 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 161/319] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:44.190380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95e799-503a-4bd8-8cc0-4311ca39252e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.190708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95e799-503a-4bd8-8cc0-4311ca39252e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.190988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95e799-503a-4bd8-8cc0-4311ca39252e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.197798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0f9bf-94ac-472e-80f0-dc2d4b5ed331 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.198089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0f9bf-94ac-472e-80f0-dc2d4b5ed331 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.198362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d0f9bf-94ac-472e-80f0-dc2d4b5ed331 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.199879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 162/320] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:11:44.204834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4aa30-372e-4287-afca-08db3548b825 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.205114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4aa30-372e-4287-afca-08db3548b825 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.205395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec4aa30-372e-4287-afca-08db3548b825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.210614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8e95e799-503a-4bd8-8cc0-4311ca39252e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:44.211827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8e95e799-503a-4bd8-8cc0-4311ca39252e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:44.223419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8ec4aa30-372e-4287-afca-08db3548b825 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:44.224693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ec4aa30-372e-4287-afca-08db3548b825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:44.286009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 159/321] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:44.292655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 163/322] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:44.308900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95e799-503a-4bd8-8cc0-4311ca39252e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:44.314141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4aa30-372e-4287-afca-08db3548b825 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:44.375216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:44.376274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:44.424365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec4aa30-372e-4287-afca-08db3548b825 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:44.424565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95e799-503a-4bd8-8cc0-4311ca39252e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:44.462771 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6001a7df-e904-4b79-a400-89b1d8821873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:44.463434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001a7df-e904-4b79-a400-89b1d8821873 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.463717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001a7df-e904-4b79-a400-89b1d8821873 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.463996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6001a7df-e904-4b79-a400-89b1d8821873 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.466074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217b9a8-4ee3-4dc6-80f3-2505f53aeb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:44.466735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b9a8-4ee3-4dc6-80f3-2505f53aeb59 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.467013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b9a8-4ee3-4dc6-80f3-2505f53aeb59 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.467287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7217b9a8-4ee3-4dc6-80f3-2505f53aeb59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.475063 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6001a7df-e904-4b79-a400-89b1d8821873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:44.477692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7217b9a8-4ee3-4dc6-80f3-2505f53aeb59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6001a7df-e904-4b79-a400-89b1d8821873 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:44.480099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:44.480920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:44.481706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 164/323] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7217b9a8-4ee3-4dc6-80f3-2505f53aeb59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:44.482883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:44.484349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:44.485366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:44.485979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc98b89-ddd2-44b8-aa96-5f5dd6d28999 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.485979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc98b89-ddd2-44b8-aa96-5f5dd6d28999 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.485979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc98b89-ddd2-44b8-aa96-5f5dd6d28999 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.486142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 160/324] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:44.487470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 165/325] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:44.487688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217b9a8-4ee3-4dc6-80f3-2505f53aeb59 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:44.490314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc98b89-ddd2-44b8-aa96-5f5dd6d28999 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:44.587081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7217b9a8-4ee3-4dc6-80f3-2505f53aeb59 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:44.597262 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc98b89-ddd2-44b8-aa96-5f5dd6d28999 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:44.607663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:44.618893 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:44.622431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2c4c1a-1a07-4688-a36e-0c12f71c17e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:44.623071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c4c1a-1a07-4688-a36e-0c12f71c17e5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.623355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c4c1a-1a07-4688-a36e-0c12f71c17e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.623622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2c4c1a-1a07-4688-a36e-0c12f71c17e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.624353 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c4c1a-1a07-4688-a36e-0c12f71c17e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:44.624682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c4c1a-1a07-4688-a36e-0c12f71c17e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:44.626115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2c4c1a-1a07-4688-a36e-0c12f71c17e5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:44.633475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf1bad-ad16-46fe-8dc7-9085c9606740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:44.634142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf1bad-ad16-46fe-8dc7-9085c9606740 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.634615 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf1bad-ad16-46fe-8dc7-9085c9606740 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.634716 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 161/326] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:44.634787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf1bad-ad16-46fe-8dc7-9085c9606740 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.635410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf1bad-ad16-46fe-8dc7-9085c9606740 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:44.635627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d8295-32d6-4def-a66c-a0027718cd51 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.635734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf1bad-ad16-46fe-8dc7-9085c9606740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:44.635918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d8295-32d6-4def-a66c-a0027718cd51 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.636194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d8295-32d6-4def-a66c-a0027718cd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.637090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cf1bad-ad16-46fe-8dc7-9085c9606740 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:44.646953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 166/327] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:44.647648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf1bad-ad16-46fe-8dc7-9085c9606740 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:44.653958 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-146d8295-32d6-4def-a66c-a0027718cd51 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:44.655156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-146d8295-32d6-4def-a66c-a0027718cd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:44.724506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 162/328] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:44.725131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d8295-32d6-4def-a66c-a0027718cd51 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:44.751168 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf1bad-ad16-46fe-8dc7-9085c9606740 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:44.785357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d82d2-d760-464b-94b5-90ff992861f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:44.786056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d82d2-d760-464b-94b5-90ff992861f8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.786348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d82d2-d760-464b-94b5-90ff992861f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.786620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d82d2-d760-464b-94b5-90ff992861f8 None admin] PATH_INFO: `/v3/users/ff54e343564b44f29ee239d821e6b440` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.787832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d82d2-d760-464b-94b5-90ff992861f8 None admin] RBAC: Authorizing `identity:get_user(user_id=ff54e343564b44f29ee239d821e6b440)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:44.789298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d82d2-d760-464b-94b5-90ff992861f8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:44.791003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 167/329] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:44 2022] GET /identity/v3/users/ff54e343564b44f29ee239d821e6b440 => generated 293 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:44.791776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d82d2-d760-464b-94b5-90ff992861f8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:44.836688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d8295-32d6-4def-a66c-a0027718cd51 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:44.847466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:44.850759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:44.872349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e755f-ce63-4bcb-a74e-4bdbb157119a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:44.872993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e755f-ce63-4bcb-a74e-4bdbb157119a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.873277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e755f-ce63-4bcb-a74e-4bdbb157119a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.873547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e755f-ce63-4bcb-a74e-4bdbb157119a None admin] PATH_INFO: `/v3/users/27a92ada0271441b8a7ceefdc2625bc3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.874785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e755f-ce63-4bcb-a74e-4bdbb157119a None admin] RBAC: Authorizing `identity:get_user(user_id=27a92ada0271441b8a7ceefdc2625bc3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:44.876221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e755f-ce63-4bcb-a74e-4bdbb157119a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:44.878087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 163/330] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:44 2022] GET /identity/v3/users/27a92ada0271441b8a7ceefdc2625bc3 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:44.878672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e755f-ce63-4bcb-a74e-4bdbb157119a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:44.900398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d82d2-d760-464b-94b5-90ff992861f8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:44.934303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c0a60-2e8c-497c-980d-0a2620cb7663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:44.934944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c0a60-2e8c-497c-980d-0a2620cb7663 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:44.935217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c0a60-2e8c-497c-980d-0a2620cb7663 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:44.935493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05c0a60-2e8c-497c-980d-0a2620cb7663 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:44.936683 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c0a60-2e8c-497c-980d-0a2620cb7663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:44.938165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05c0a60-2e8c-497c-980d-0a2620cb7663 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:44.939762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 168/331] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:11:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:44.940596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c0a60-2e8c-497c-980d-0a2620cb7663 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:44.985179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e755f-ce63-4bcb-a74e-4bdbb157119a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:45.021195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58361b-a48a-449d-9f19-cbb7724d897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:45.021904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58361b-a48a-449d-9f19-cbb7724d897c None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:45.022161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58361b-a48a-449d-9f19-cbb7724d897c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:45.022469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58361b-a48a-449d-9f19-cbb7724d897c None admin] PATH_INFO: `/v3/system/users/ff54e343564b44f29ee239d821e6b440/roles/511279b616c245e2baad4d3a59cd477e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:45.023981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58361b-a48a-449d-9f19-cbb7724d897c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ff54e343564b44f29ee239d821e6b440, role_id=511279b616c245e2baad4d3a59cd477e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:45.025372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58361b-a48a-449d-9f19-cbb7724d897c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:45.033318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 164/332] 10.222.0.39 () {58 vars in 1334 bytes} [Tue Jul 26 07:11:44 2022] PUT /identity/v3/system/users/ff54e343564b44f29ee239d821e6b440/roles/511279b616c245e2baad4d3a59cd477e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:45.033962 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58361b-a48a-449d-9f19-cbb7724d897c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:45.042696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05c0a60-2e8c-497c-980d-0a2620cb7663 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:45.063481 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:45.088597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636fcd03-5da7-415d-9289-811bac7ac65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:45.089249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fcd03-5da7-415d-9289-811bac7ac65d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:45.089528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fcd03-5da7-415d-9289-811bac7ac65d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:45.090020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:45.090672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636fcd03-5da7-415d-9289-811bac7ac65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:45.091652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fcd03-5da7-415d-9289-811bac7ac65d None admin] RBAC: Adding query filter params (role.id=64dfc66a8abe4468ac4f1ca071d28bd4, user.id=27a92ada0271441b8a7ceefdc2625bc3, scope.system=all) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:45.092128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fcd03-5da7-415d-9289-811bac7ac65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:45.093967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636fcd03-5da7-415d-9289-811bac7ac65d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:45.121780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 169/333] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Jul 26 07:11:44 2022] GET /identity/v3/role_assignments?user.id=27a92ada0271441b8a7ceefdc2625bc3&scope.system=all&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:45.177139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58361b-a48a-449d-9f19-cbb7724d897c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:45.225586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012b984c-3cfe-4eba-9625-8840c311e56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:45.226304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b984c-3cfe-4eba-9625-8840c311e56e None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:45.226586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b984c-3cfe-4eba-9625-8840c311e56e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:45.226864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012b984c-3cfe-4eba-9625-8840c311e56e None admin] PATH_INFO: `/v3/groups` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:45.227562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b984c-3cfe-4eba-9625-8840c311e56e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:45.229056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012b984c-3cfe-4eba-9625-8840c311e56e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:45.247473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 165/334] 10.222.0.39 () {60 vars in 1134 bytes} [Tue Jul 26 07:11:45 2022] POST /identity/v3/groups => generated 226 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:11:47.365060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e66382-3636-4daf-a69b-99c7af5df8b1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:47.366011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e66382-3636-4daf-a69b-99c7af5df8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:47.366488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e66382-3636-4daf-a69b-99c7af5df8b1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:47.368042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 170/335] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:47.373821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff3a42-78b1-4ff4-a694-6aa860987d82 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:47.374668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff3a42-78b1-4ff4-a694-6aa860987d82 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:47.375108 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aff3a42-78b1-4ff4-a694-6aa860987d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:47.402764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4aff3a42-78b1-4ff4-a694-6aa860987d82 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:47.404276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4aff3a42-78b1-4ff4-a694-6aa860987d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:47.511946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 166/336] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:47.544815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e66382-3636-4daf-a69b-99c7af5df8b1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:47.587590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a46c9-1fae-4b8c-9897-8947fa552933 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:47.588082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a46c9-1fae-4b8c-9897-8947fa552933 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:47.588537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a46c9-1fae-4b8c-9897-8947fa552933 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:47.591499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 167/337] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:11:47.600339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d4b73-f905-4f0b-a5da-1575005a4629 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:47.600920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d4b73-f905-4f0b-a5da-1575005a4629 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:47.601398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d4b73-f905-4f0b-a5da-1575005a4629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:47.635477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-763d4b73-f905-4f0b-a5da-1575005a4629 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:47.636632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-763d4b73-f905-4f0b-a5da-1575005a4629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:47.684507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:47.744280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 168/338] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:47.764553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e66382-3636-4daf-a69b-99c7af5df8b1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:47.847761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf2ff4-ed6e-48b4-b091-1ce57113bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:47.848901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf2ff4-ed6e-48b4-b091-1ce57113bcfe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:47.849412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf2ff4-ed6e-48b4-b091-1ce57113bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:47.849977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cf2ff4-ed6e-48b4-b091-1ce57113bcfe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:47.852550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d4b73-f905-4f0b-a5da-1575005a4629 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:47.881999 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cf2ff4-ed6e-48b4-b091-1ce57113bcfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-93cf2ff4-ed6e-48b4-b091-1ce57113bcfe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:47.901514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:47.902500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 26 07:11:47.903317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:47.906949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 171/339] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:47.917223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf2ff4-ed6e-48b4-b091-1ce57113bcfe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:48.020753 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:48.085375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d4b73-f905-4f0b-a5da-1575005a4629 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:48.143833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190036d-504c-4f13-a27d-ac84a491634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:48.144534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190036d-504c-4f13-a27d-ac84a491634f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:48.149767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190036d-504c-4f13-a27d-ac84a491634f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:48.149767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190036d-504c-4f13-a27d-ac84a491634f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:48.167928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190036d-504c-4f13-a27d-ac84a491634f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2190036d-504c-4f13-a27d-ac84a491634f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:48.173513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:48.175389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:48.177068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:48.179424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 169/340] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:48.184876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190036d-504c-4f13-a27d-ac84a491634f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:48.220010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:48.227079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cf2ff4-ed6e-48b4-b091-1ce57113bcfe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:48.314769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c23416-90ca-4403-a903-32e21f950f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:48.315468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23416-90ca-4403-a903-32e21f950f44 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:48.315915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23416-90ca-4403-a903-32e21f950f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:48.316214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c23416-90ca-4403-a903-32e21f950f44 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:48.316963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23416-90ca-4403-a903-32e21f950f44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:48.321715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23416-90ca-4403-a903-32e21f950f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:48.321715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c23416-90ca-4403-a903-32e21f950f44 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:48.334701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 172/341] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:48.340863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c23416-90ca-4403-a903-32e21f950f44 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:48.431322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190036d-504c-4f13-a27d-ac84a491634f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:48.492463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef1d75a-eb00-4850-a49a-c6bac4afa123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:48.493367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1d75a-eb00-4850-a49a-c6bac4afa123 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:48.493834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1d75a-eb00-4850-a49a-c6bac4afa123 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:48.494231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1d75a-eb00-4850-a49a-c6bac4afa123 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:48.496218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1d75a-eb00-4850-a49a-c6bac4afa123 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:48.496695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1d75a-eb00-4850-a49a-c6bac4afa123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:48.498509 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef1d75a-eb00-4850-a49a-c6bac4afa123 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:48.509049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 170/342] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:48.510822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f86cd-2a98-441f-919c-d82df1995894 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:48.511266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f86cd-2a98-441f-919c-d82df1995894 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:48.511677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f86cd-2a98-441f-919c-d82df1995894 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:48.513274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 171/343] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:48.516038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287321-e9d6-484e-aafa-a25cf37f1999 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:48.516604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287321-e9d6-484e-aafa-a25cf37f1999 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:48.517069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61287321-e9d6-484e-aafa-a25cf37f1999 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:48.519492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 172/344] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:48.520683 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287321-e9d6-484e-aafa-a25cf37f1999 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:48.542594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c23416-90ca-4403-a903-32e21f950f44 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:48.545870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:48.594319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a993d12-fe79-4057-86c9-8416d134dd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:48.595393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993d12-fe79-4057-86c9-8416d134dd29 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:48.595863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993d12-fe79-4057-86c9-8416d134dd29 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:48.596314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a993d12-fe79-4057-86c9-8416d134dd29 None admin] PATH_INFO: `/v3/users/ff54e343564b44f29ee239d821e6b440` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:48.597938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a993d12-fe79-4057-86c9-8416d134dd29 None admin] RBAC: Authorizing `identity:get_user(user_id=ff54e343564b44f29ee239d821e6b440)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:48.599850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a993d12-fe79-4057-86c9-8416d134dd29 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:48.602209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 173/345] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:11:48 2022] GET /identity/v3/users/ff54e343564b44f29ee239d821e6b440 => generated 293 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:48.603736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff5e4c-c15b-4866-8cba-4fa3eb10c20c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:48.604204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff5e4c-c15b-4866-8cba-4fa3eb10c20c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:48.604717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bff5e4c-c15b-4866-8cba-4fa3eb10c20c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:48.627021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5bff5e4c-c15b-4866-8cba-4fa3eb10c20c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:48.628726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bff5e4c-c15b-4866-8cba-4fa3eb10c20c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:48.631833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:48.652949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61287321-e9d6-484e-aafa-a25cf37f1999 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:48.695618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f102c-9330-45d6-94a6-5e9923b34466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:48.696529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f102c-9330-45d6-94a6-5e9923b34466 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:48.696998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f102c-9330-45d6-94a6-5e9923b34466 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:48.697409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f102c-9330-45d6-94a6-5e9923b34466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:48.743558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 174/346] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:48.746467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44e43a-a94d-43b7-9288-cb334ccc85b2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:48.747496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44e43a-a94d-43b7-9288-cb334ccc85b2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:48.748274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44e43a-a94d-43b7-9288-cb334ccc85b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:48.779078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f102c-9330-45d6-94a6-5e9923b34466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:48.780064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5c44e43a-a94d-43b7-9288-cb334ccc85b2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:48.781218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f102c-9330-45d6-94a6-5e9923b34466 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:48.782590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5c44e43a-a94d-43b7-9288-cb334ccc85b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:48.889250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:48.904515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 173/347] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:48.906068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f102c-9330-45d6-94a6-5e9923b34466 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:48.921385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 175/348] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:48.924252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44e43a-a94d-43b7-9288-cb334ccc85b2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:48.977418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:49.106306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f102c-9330-45d6-94a6-5e9923b34466 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:49.120749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44e43a-a94d-43b7-9288-cb334ccc85b2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:49.175945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b77fb3-2390-428c-a58d-876cc0216b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:49.177620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77fb3-2390-428c-a58d-876cc0216b74 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:49.178539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77fb3-2390-428c-a58d-876cc0216b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:49.179320 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b77fb3-2390-428c-a58d-876cc0216b74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:49.181098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b77fb3-2390-428c-a58d-876cc0216b74 None admin] RBAC: Adding query filter params (role.id=511279b616c245e2baad4d3a59cd477e, user.id=ff54e343564b44f29ee239d821e6b440, scope.system=all) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:49.181984 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b77fb3-2390-428c-a58d-876cc0216b74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:49.185005 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b77fb3-2390-428c-a58d-876cc0216b74 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:49.188987 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdab1dc-72c1-495d-9723-afe680331058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:49.190114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdab1dc-72c1-495d-9723-afe680331058 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:49.190627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdab1dc-72c1-495d-9723-afe680331058 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:49.191684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdab1dc-72c1-495d-9723-afe680331058 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:49.211277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdab1dc-72c1-495d-9723-afe680331058 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:49.214435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 174/349] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Jul 26 07:11:48 2022] GET /identity/v3/role_assignments?user.id=ff54e343564b44f29ee239d821e6b440&scope.system=all&role.id=511279b616c245e2baad4d3a59cd477e => generated 510 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:49.215340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b77fb3-2390-428c-a58d-876cc0216b74 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-dfdab1dc-72c1-495d-9723-afe680331058 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:49.220081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:49.221990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 176/350] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:49.224338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdab1dc-72c1-495d-9723-afe680331058 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:49.328205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:49.422184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:49.446842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdab1dc-72c1-495d-9723-afe680331058 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:49.463295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b77fb3-2390-428c-a58d-876cc0216b74 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:49.515655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483d005-58fa-47fa-8fae-5adea364f7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:49.517150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483d005-58fa-47fa-8fae-5adea364f7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:49.517657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483d005-58fa-47fa-8fae-5adea364f7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:49.518926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483d005-58fa-47fa-8fae-5adea364f7a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:49.531903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab8befd-d416-41d3-8a2e-83c9bc02711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:49.532595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8befd-d416-41d3-8a2e-83c9bc02711d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:49.533031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8befd-d416-41d3-8a2e-83c9bc02711d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:49.533306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab8befd-d416-41d3-8a2e-83c9bc02711d None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:49.534772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8befd-d416-41d3-8a2e-83c9bc02711d None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:49.536916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab8befd-d416-41d3-8a2e-83c9bc02711d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:49.539154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 175/351] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:49 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:49.540426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9615756-072d-43d3-af9a-d9b332b969cd None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:49.540854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9615756-072d-43d3-af9a-d9b332b969cd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:49.541136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9615756-072d-43d3-af9a-d9b332b969cd None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:49.541479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3483d005-58fa-47fa-8fae-5adea364f7a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:49.542852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 176/352] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:49.543409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9615756-072d-43d3-af9a-d9b332b969cd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3483d005-58fa-47fa-8fae-5adea364f7a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:49.552971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:49.554991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 177/353] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:49.558902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd097b0-374d-44af-8075-8dce5575ef3b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:49.561399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd097b0-374d-44af-8075-8dce5575ef3b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:49.561399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd097b0-374d-44af-8075-8dce5575ef3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:49.588517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cdd097b0-374d-44af-8075-8dce5575ef3b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:49.589779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cdd097b0-374d-44af-8075-8dce5575ef3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:49.704904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 178/354] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:49.710232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd097b0-374d-44af-8075-8dce5575ef3b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:49.755876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9615756-072d-43d3-af9a-d9b332b969cd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:49.773016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:49.828270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a91c21-cb33-4d32-95ca-9ef66d1e7fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:49.829350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a91c21-cb33-4d32-95ca-9ef66d1e7fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:49.829854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a91c21-cb33-4d32-95ca-9ef66d1e7fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:49.830305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a91c21-cb33-4d32-95ca-9ef66d1e7fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:49.831675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a91c21-cb33-4d32-95ca-9ef66d1e7fb2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:49.832183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a91c21-cb33-4d32-95ca-9ef66d1e7fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:49.834889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a91c21-cb33-4d32-95ca-9ef66d1e7fb2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:49.849994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 177/355] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:49.851071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a91c21-cb33-4d32-95ca-9ef66d1e7fb2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:49.869472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:49.898137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd097b0-374d-44af-8075-8dce5575ef3b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:49.950508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c06ae-2745-4ab0-bd15-a893ef7eb635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:49.951183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c06ae-2745-4ab0-bd15-a893ef7eb635 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:49.951455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c06ae-2745-4ab0-bd15-a893ef7eb635 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:49.951727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538c06ae-2745-4ab0-bd15-a893ef7eb635 None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:49.966057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c06ae-2745-4ab0-bd15-a893ef7eb635 None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:49.967559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538c06ae-2745-4ab0-bd15-a893ef7eb635 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:49.969633 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 179/356] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:11:49 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:49.970438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c06ae-2745-4ab0-bd15-a893ef7eb635 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:50.041135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a91c21-cb33-4d32-95ca-9ef66d1e7fb2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:50.099118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67188ac5-3944-4148-b05e-6974ec5971c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:50.099770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67188ac5-3944-4148-b05e-6974ec5971c0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:50.100112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67188ac5-3944-4148-b05e-6974ec5971c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:50.100367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67188ac5-3944-4148-b05e-6974ec5971c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:50.101056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67188ac5-3944-4148-b05e-6974ec5971c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:50.101357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67188ac5-3944-4148-b05e-6974ec5971c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:50.102884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67188ac5-3944-4148-b05e-6974ec5971c0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:50.112851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 178/357] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:50.120007 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55870e-3ca2-4de6-882d-9112708dc0eb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:50.120292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55870e-3ca2-4de6-882d-9112708dc0eb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:50.120551 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55870e-3ca2-4de6-882d-9112708dc0eb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:50.122027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 179/358] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:50.122809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55870e-3ca2-4de6-882d-9112708dc0eb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:50.137562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538c06ae-2745-4ab0-bd15-a893ef7eb635 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:50.184911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:50.199413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe9655-3e72-4a1a-863b-c3e2ad45a6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:50.200243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe9655-3e72-4a1a-863b-c3e2ad45a6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:50.200699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe9655-3e72-4a1a-863b-c3e2ad45a6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:50.201106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe9655-3e72-4a1a-863b-c3e2ad45a6c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:50.217203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:50.224100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abe9655-3e72-4a1a-863b-c3e2ad45a6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3abe9655-3e72-4a1a-863b-c3e2ad45a6c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:50.234088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:50.235957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:50.237535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 180/359] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:50.244004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe9655-3e72-4a1a-863b-c3e2ad45a6c2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:50.301390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55870e-3ca2-4de6-882d-9112708dc0eb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:50.359001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b13db4a-f165-4ae4-af76-9749049ebc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:50.360832 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13db4a-f165-4ae4-af76-9749049ebc49 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:50.361645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13db4a-f165-4ae4-af76-9749049ebc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:50.362166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b13db4a-f165-4ae4-af76-9749049ebc49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:50.434323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe9655-3e72-4a1a-863b-c3e2ad45a6c2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:50.495815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2b349-2da3-44af-9be4-0c8cd6157c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:50.496294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13db4a-f165-4ae4-af76-9749049ebc49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:50.496905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2b349-2da3-44af-9be4-0c8cd6157c9e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:50.497220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2b349-2da3-44af-9be4-0c8cd6157c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:50.497645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d2b349-2da3-44af-9be4-0c8cd6157c9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:50.498832 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b13db4a-f165-4ae4-af76-9749049ebc49 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:50.499125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2b349-2da3-44af-9be4-0c8cd6157c9e None admin] RBAC: Adding query filter params (group.id=8577807d221144cba4cc471a3d22db2c, role.id=64dfc66a8abe4468ac4f1ca071d28bd4, scope.project.id=9ad9190f63064d308a0423e34054b7df) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:50.499559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2b349-2da3-44af-9be4-0c8cd6157c9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:50.502223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d2b349-2da3-44af-9be4-0c8cd6157c9e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:50.512152 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 181/360] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jul 26 07:11:50 2022] GET /identity/v3/role_assignments?group.id=8577807d221144cba4cc471a3d22db2c&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 260 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:50.513338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2b349-2da3-44af-9be4-0c8cd6157c9e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:50.581793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:50.625444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:50.686662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d2b349-2da3-44af-9be4-0c8cd6157c9e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:50.716339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 180/361] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:50.719081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7575a7-52d7-43a4-abfc-6bb051a8205e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:50.719926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7575a7-52d7-43a4-abfc-6bb051a8205e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:50.720703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7575a7-52d7-43a4-abfc-6bb051a8205e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:50.747302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048f7a7-6093-4ddb-b366-0ec87e53a643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:50.748794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048f7a7-6093-4ddb-b366-0ec87e53a643 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:50.749497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048f7a7-6093-4ddb-b366-0ec87e53a643 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:50.749648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3a7575a7-52d7-43a4-abfc-6bb051a8205e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:50.750477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048f7a7-6093-4ddb-b366-0ec87e53a643 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:50.751527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a7575a7-52d7-43a4-abfc-6bb051a8205e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:50.873036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 181/362] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:50.875161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7575a7-52d7-43a4-abfc-6bb051a8205e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:50.884834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048f7a7-6093-4ddb-b366-0ec87e53a643 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:50.887621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048f7a7-6093-4ddb-b366-0ec87e53a643 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:51.008609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:51.010271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:51.078375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7575a7-52d7-43a4-abfc-6bb051a8205e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:51.101427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 182/363] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:51.103035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048f7a7-6093-4ddb-b366-0ec87e53a643 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:51.143035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44b419-8f69-4c20-8c71-322a70f0555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:51.144081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44b419-8f69-4c20-8c71-322a70f0555f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:51.144495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44b419-8f69-4c20-8c71-322a70f0555f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:51.144913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c44b419-8f69-4c20-8c71-322a70f0555f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:51.147091 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44b419-8f69-4c20-8c71-322a70f0555f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:51.147566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44b419-8f69-4c20-8c71-322a70f0555f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:51.149111 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c44b419-8f69-4c20-8c71-322a70f0555f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:51.168467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 182/364] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:51.170356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44b419-8f69-4c20-8c71-322a70f0555f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:51.303899 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048f7a7-6093-4ddb-b366-0ec87e53a643 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:51.363038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c44b419-8f69-4c20-8c71-322a70f0555f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:51.376800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b267c-67a0-4de0-a38f-31eb8864d90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:51.377757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b267c-67a0-4de0-a38f-31eb8864d90e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:51.378228 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b267c-67a0-4de0-a38f-31eb8864d90e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:51.378624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b267c-67a0-4de0-a38f-31eb8864d90e None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:51.380500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b267c-67a0-4de0-a38f-31eb8864d90e None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:51.382963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b267c-67a0-4de0-a38f-31eb8864d90e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:51.386165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 183/365] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:51 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:51.387577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b267c-67a0-4de0-a38f-31eb8864d90e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:51.422012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:51.445642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a2813-e5dd-4afa-aa1a-4cdd6e0f0a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:51.446791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a2813-e5dd-4afa-aa1a-4cdd6e0f0a05 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:51.447281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a2813-e5dd-4afa-aa1a-4cdd6e0f0a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:51.447755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a2813-e5dd-4afa-aa1a-4cdd6e0f0a05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:51.450816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:51.477992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7a2813-e5dd-4afa-aa1a-4cdd6e0f0a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2f7a2813-e5dd-4afa-aa1a-4cdd6e0f0a05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:51.489314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:51.490841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:51.492748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 183/366] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:11:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:51.494661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a2813-e5dd-4afa-aa1a-4cdd6e0f0a05 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:51.595940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b267c-67a0-4de0-a38f-31eb8864d90e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:51.658873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8de30-6c7c-4b77-b2bb-2ccea32d307f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:51.660668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8de30-6c7c-4b77-b2bb-2ccea32d307f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:51.661167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8de30-6c7c-4b77-b2bb-2ccea32d307f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:51.661651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8de30-6c7c-4b77-b2bb-2ccea32d307f None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:51.664717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e8de30-6c7c-4b77-b2bb-2ccea32d307f None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:51.668239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e8de30-6c7c-4b77-b2bb-2ccea32d307f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:51.673489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 184/367] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:51 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:51.673852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8de30-6c7c-4b77-b2bb-2ccea32d307f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:51.694740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a2813-e5dd-4afa-aa1a-4cdd6e0f0a05 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:51.745473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71618a-ce4a-4d17-bdb0-4da39126c979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:51.746188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71618a-ce4a-4d17-bdb0-4da39126c979 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:51.746460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71618a-ce4a-4d17-bdb0-4da39126c979 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:51.746743 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c71618a-ce4a-4d17-bdb0-4da39126c979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:51.847075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:51.873633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71618a-ce4a-4d17-bdb0-4da39126c979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:51.874469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:51.877452 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c71618a-ce4a-4d17-bdb0-4da39126c979 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:51.879492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8de30-6c7c-4b77-b2bb-2ccea32d307f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:51.954566 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14185f-772e-4d38-b222-1f77f4eef30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:51.955334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14185f-772e-4d38-b222-1f77f4eef30b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:51.955820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14185f-772e-4d38-b222-1f77f4eef30b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:51.956407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14185f-772e-4d38-b222-1f77f4eef30b None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:51.958720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14185f-772e-4d38-b222-1f77f4eef30b None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:51.961641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b14185f-772e-4d38-b222-1f77f4eef30b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:51.970415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 185/368] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:11:51 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:51.972168 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14185f-772e-4d38-b222-1f77f4eef30b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:52.027451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 184/369] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:52.035950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71618a-ce4a-4d17-bdb0-4da39126c979 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:52.149830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14185f-772e-4d38-b222-1f77f4eef30b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:52.187653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:52.194873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb772f-cbfb-4f80-b069-bfbad0cefcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:52.195818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb772f-cbfb-4f80-b069-bfbad0cefcac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:52.196094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb772f-cbfb-4f80-b069-bfbad0cefcac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:52.196363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bb772f-cbfb-4f80-b069-bfbad0cefcac None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:52.197078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb772f-cbfb-4f80-b069-bfbad0cefcac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:52.197553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb772f-cbfb-4f80-b069-bfbad0cefcac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:52.199333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bb772f-cbfb-4f80-b069-bfbad0cefcac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:52.209241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c71618a-ce4a-4d17-bdb0-4da39126c979 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:52.215658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 186/370] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:52.217218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb772f-cbfb-4f80-b069-bfbad0cefcac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:52.268311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:52.270717 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8b58f-7909-47d4-9a78-dead77261b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:52.271471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8b58f-7909-47d4-9a78-dead77261b36 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:52.271723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8b58f-7909-47d4-9a78-dead77261b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:52.272350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8b58f-7909-47d4-9a78-dead77261b36 None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:52.273375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8b58f-7909-47d4-9a78-dead77261b36 None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:52.275013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8b58f-7909-47d4-9a78-dead77261b36 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:52.286612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 185/371] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:11:52 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:52.286612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8b58f-7909-47d4-9a78-dead77261b36 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:52.395136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bb772f-cbfb-4f80-b069-bfbad0cefcac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:52.449377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8b58f-7909-47d4-9a78-dead77261b36 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:52.456180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458aa0a4-30f1-41da-a1f4-b37a9d59143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:52.457435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa0a4-30f1-41da-a1f4-b37a9d59143d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:52.458090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa0a4-30f1-41da-a1f4-b37a9d59143d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:52.458716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458aa0a4-30f1-41da-a1f4-b37a9d59143d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:52.460448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aa0a4-30f1-41da-a1f4-b37a9d59143d None admin] RBAC: Adding query filter params (group.id=8577807d221144cba4cc471a3d22db2c, role.id=5e0c08646046426385776839f1f168cd, scope.project.id=9ad9190f63064d308a0423e34054b7df) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:52.461053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aa0a4-30f1-41da-a1f4-b37a9d59143d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:52.464001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458aa0a4-30f1-41da-a1f4-b37a9d59143d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:52.481785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 187/372] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jul 26 07:11:52 2022] GET /identity/v3/role_assignments?group.id=8577807d221144cba4cc471a3d22db2c&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=5e0c08646046426385776839f1f168cd => generated 260 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:52.481785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458aa0a4-30f1-41da-a1f4-b37a9d59143d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:52.496568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16627f9-6221-43c6-b378-71d76f6a8ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:52.497613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16627f9-6221-43c6-b378-71d76f6a8ebf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:52.498102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16627f9-6221-43c6-b378-71d76f6a8ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:52.498584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16627f9-6221-43c6-b378-71d76f6a8ebf None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:52.501363 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16627f9-6221-43c6-b378-71d76f6a8ebf None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:52.503935 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16627f9-6221-43c6-b378-71d76f6a8ebf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:52.508781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 186/373] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:52 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:52.510566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16627f9-6221-43c6-b378-71d76f6a8ebf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:52.570231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:52.707238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:52.721821 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458aa0a4-30f1-41da-a1f4-b37a9d59143d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:52.730278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16627f9-6221-43c6-b378-71d76f6a8ebf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:52.781001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2c08c-17db-4a1c-8b2c-8863243060b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:52.782014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2c08c-17db-4a1c-8b2c-8863243060b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:52.782445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2c08c-17db-4a1c-8b2c-8863243060b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:52.782868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2c08c-17db-4a1c-8b2c-8863243060b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:52.800229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cde85f-0e2a-41a0-8ba4-0f64c60b7531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:52.801852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cde85f-0e2a-41a0-8ba4-0f64c60b7531 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:52.802645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cde85f-0e2a-41a0-8ba4-0f64c60b7531 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:52.803387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cde85f-0e2a-41a0-8ba4-0f64c60b7531 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:52.805508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cde85f-0e2a-41a0-8ba4-0f64c60b7531 None admin] RBAC: Adding query filter params (group.id=8577807d221144cba4cc471a3d22db2c, role.id=64dfc66a8abe4468ac4f1ca071d28bd4, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:52.806524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cde85f-0e2a-41a0-8ba4-0f64c60b7531 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:52.809679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cde85f-0e2a-41a0-8ba4-0f64c60b7531 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:52.822956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 187/374] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jul 26 07:11:52 2022] GET /identity/v3/role_assignments?group.id=8577807d221144cba4cc471a3d22db2c&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 260 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:52.823999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cde85f-0e2a-41a0-8ba4-0f64c60b7531 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:52.872620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2c08c-17db-4a1c-8b2c-8863243060b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:52.874199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2c08c-17db-4a1c-8b2c-8863243060b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:52.948216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:52.963418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cde85f-0e2a-41a0-8ba4-0f64c60b7531 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:52.978915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:52.989911 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 188/375] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:11:52.995839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2c08c-17db-4a1c-8b2c-8863243060b6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:53.017112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b536b7-6542-45b1-a9cf-cc1612a281c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:53.018221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b536b7-6542-45b1-a9cf-cc1612a281c4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:53.020900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b536b7-6542-45b1-a9cf-cc1612a281c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:53.021490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b536b7-6542-45b1-a9cf-cc1612a281c4 None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:53.024736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b536b7-6542-45b1-a9cf-cc1612a281c4 None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:53.028384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b536b7-6542-45b1-a9cf-cc1612a281c4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:53.034037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 188/376] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:11:52 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:53.039964 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b536b7-6542-45b1-a9cf-cc1612a281c4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:53.184777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2c08c-17db-4a1c-8b2c-8863243060b6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:53.229554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d36ac4-199d-4768-aeac-acd1f27e572a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:53.230238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d36ac4-199d-4768-aeac-acd1f27e572a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:53.230527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d36ac4-199d-4768-aeac-acd1f27e572a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:53.230809 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d36ac4-199d-4768-aeac-acd1f27e572a None admin] PATH_INFO: `/v3/projects/9aaf3bcccded4ffcb2a432d5c86ce8ea` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:53.232680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d36ac4-199d-4768-aeac-acd1f27e572a None admin] RBAC: Authorizing `identity:get_project(project_id=9aaf3bcccded4ffcb2a432d5c86ce8ea)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:53.234552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d36ac4-199d-4768-aeac-acd1f27e572a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:53.237057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 189/377] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:52 2022] GET /identity/v3/projects/9aaf3bcccded4ffcb2a432d5c86ce8ea => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:53.238696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0663a-17d5-413b-9cfa-859d7b8bfcf4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:53.238987 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0663a-17d5-413b-9cfa-859d7b8bfcf4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:53.239265 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0663a-17d5-413b-9cfa-859d7b8bfcf4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:53.245693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 190/378] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:53.246896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0663a-17d5-413b-9cfa-859d7b8bfcf4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:53.288750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b536b7-6542-45b1-a9cf-cc1612a281c4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:53.331718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d246cf6c-2188-4f78-b289-6161f09b91e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:53.332423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246cf6c-2188-4f78-b289-6161f09b91e7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:53.332709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246cf6c-2188-4f78-b289-6161f09b91e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:53.333024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d246cf6c-2188-4f78-b289-6161f09b91e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:53.334040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246cf6c-2188-4f78-b289-6161f09b91e7 None admin] RBAC: Adding query filter params (group.id=8577807d221144cba4cc471a3d22db2c, role.id=5e0c08646046426385776839f1f168cd, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:53.334567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246cf6c-2188-4f78-b289-6161f09b91e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:53.336048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d246cf6c-2188-4f78-b289-6161f09b91e7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:53.350403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 189/379] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jul 26 07:11:53 2022] GET /identity/v3/role_assignments?group.id=8577807d221144cba4cc471a3d22db2c&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=5e0c08646046426385776839f1f168cd => generated 260 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:53.352037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68279b-13cd-414a-8aed-f8cd7b7d1fe4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:53.352330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68279b-13cd-414a-8aed-f8cd7b7d1fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:53.352608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68279b-13cd-414a-8aed-f8cd7b7d1fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:53.353695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:53.395913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8c68279b-13cd-414a-8aed-f8cd7b7d1fe4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:53.397304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c68279b-13cd-414a-8aed-f8cd7b7d1fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:53.432430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:53.456401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0663a-17d5-413b-9cfa-859d7b8bfcf4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:53.536968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879bac5-c11a-4952-839e-d5600ea7423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:53.538302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879bac5-c11a-4952-839e-d5600ea7423f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:53.538832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879bac5-c11a-4952-839e-d5600ea7423f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:53.539744 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3879bac5-c11a-4952-839e-d5600ea7423f None admin] PATH_INFO: `/v3/groups/e2e659b5a94d4f6199595897bbde4c95` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:53.560656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879bac5-c11a-4952-839e-d5600ea7423f None admin] RBAC: Authorizing `identity:get_group(group_id=e2e659b5a94d4f6199595897bbde4c95)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:53.564693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3879bac5-c11a-4952-839e-d5600ea7423f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:53.571749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 191/380] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:11:53 2022] GET /identity/v3/groups/e2e659b5a94d4f6199595897bbde4c95 => generated 229 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:53.582978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879bac5-c11a-4952-839e-d5600ea7423f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:53.590570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 190/381] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:53.614348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68279b-13cd-414a-8aed-f8cd7b7d1fe4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:53.844791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68279b-13cd-414a-8aed-f8cd7b7d1fe4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:53.880947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3879bac5-c11a-4952-839e-d5600ea7423f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:53.931375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:53.948008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c01fc-5a79-458f-a8ca-3dc9782d91b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:53.948601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1ce68-9163-4d99-b4c2-77abc5bc5afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:53.948733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c01fc-5a79-458f-a8ca-3dc9782d91b7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:53.948960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c01fc-5a79-458f-a8ca-3dc9782d91b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:53.949232 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c01fc-5a79-458f-a8ca-3dc9782d91b7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:53.949527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1ce68-9163-4d99-b4c2-77abc5bc5afe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:53.950011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1ce68-9163-4d99-b4c2-77abc5bc5afe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:53.950430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1ce68-9163-4d99-b4c2-77abc5bc5afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:53.952136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1ce68-9163-4d99-b4c2-77abc5bc5afe None admin] RBAC: Adding query filter params (group.id=e2e659b5a94d4f6199595897bbde4c95, role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=9aaf3bcccded4ffcb2a432d5c86ce8ea) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:53.952618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1ce68-9163-4d99-b4c2-77abc5bc5afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:53.955016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1ce68-9163-4d99-b4c2-77abc5bc5afe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:53.966671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 192/382] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jul 26 07:11:53 2022] GET /identity/v3/role_assignments?group.id=e2e659b5a94d4f6199595897bbde4c95&scope.project.id=9aaf3bcccded4ffcb2a432d5c86ce8ea&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 260 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:53.983068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c01fc-5a79-458f-a8ca-3dc9782d91b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9a4c01fc-5a79-458f-a8ca-3dc9782d91b7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:53.993027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:53.994118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:53.994980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:53.995627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 191/383] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:54.002870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1ce68-9163-4d99-b4c2-77abc5bc5afe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:54.244770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1ce68-9163-4d99-b4c2-77abc5bc5afe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:54.312188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae325a-15b4-47e4-80c9-3a2ffcf3c5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:54.313287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae325a-15b4-47e4-80c9-3a2ffcf3c5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:54.313834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae325a-15b4-47e4-80c9-3a2ffcf3c5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:54.314339 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecae325a-15b4-47e4-80c9-3a2ffcf3c5f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:54.315742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae325a-15b4-47e4-80c9-3a2ffcf3c5f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:54.316314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae325a-15b4-47e4-80c9-3a2ffcf3c5f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:54.319023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecae325a-15b4-47e4-80c9-3a2ffcf3c5f1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:54.336701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 193/384] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:54.343355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c01fc-5a79-458f-a8ca-3dc9782d91b7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:54.368408 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:54.604454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c01fc-5a79-458f-a8ca-3dc9782d91b7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:54.674355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca77610-a849-4a45-8721-a4b31bffaabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:54.675558 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca77610-a849-4a45-8721-a4b31bffaabf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:54.676064 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca77610-a849-4a45-8721-a4b31bffaabf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:54.676557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca77610-a849-4a45-8721-a4b31bffaabf None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:54.678848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca77610-a849-4a45-8721-a4b31bffaabf None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:54.681680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca77610-a849-4a45-8721-a4b31bffaabf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:54.685094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 192/385] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:11:54 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:54.691705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae325a-15b4-47e4-80c9-3a2ffcf3c5f1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:54.804380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:54.895281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecae325a-15b4-47e4-80c9-3a2ffcf3c5f1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:54.960279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1fd786-f2ec-4c96-9a51-930440eae7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:54.961410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fd786-f2ec-4c96-9a51-930440eae7ae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:54.962037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fd786-f2ec-4c96-9a51-930440eae7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:54.962538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1fd786-f2ec-4c96-9a51-930440eae7ae None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:54.964789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1fd786-f2ec-4c96-9a51-930440eae7ae None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:54.967592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1fd786-f2ec-4c96-9a51-930440eae7ae None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:54.971352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 194/386] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:54 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:54.979607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca77610-a849-4a45-8721-a4b31bffaabf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:55.138196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:55.166877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca77610-a849-4a45-8721-a4b31bffaabf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:55.212751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea92969-f8dd-41fe-906c-7072c13f0db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:55.213447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea92969-f8dd-41fe-906c-7072c13f0db5 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:55.213726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea92969-f8dd-41fe-906c-7072c13f0db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:55.214044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea92969-f8dd-41fe-906c-7072c13f0db5 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df/groups/8577807d221144cba4cc471a3d22db2c/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:55.219331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea92969-f8dd-41fe-906c-7072c13f0db5 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ad9190f63064d308a0423e34054b7df, group_id=8577807d221144cba4cc471a3d22db2c, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:55.220788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea92969-f8dd-41fe-906c-7072c13f0db5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:55.246909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 193/387] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Jul 26 07:11:54 2022] PUT /identity/v3/projects/9ad9190f63064d308a0423e34054b7df/groups/8577807d221144cba4cc471a3d22db2c/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:55.447007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff82510-db15-4c11-8388-9c3f9a7c7b3b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:55.447658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff82510-db15-4c11-8388-9c3f9a7c7b3b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:55.448165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff82510-db15-4c11-8388-9c3f9a7c7b3b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:55.450053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 195/388] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:55.460267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba16f-2bd7-4824-ae02-432ce1ee95ea None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:55.460910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba16f-2bd7-4824-ae02-432ce1ee95ea None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:55.461372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ba16f-2bd7-4824-ae02-432ce1ee95ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:55.488166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f81ba16f-2bd7-4824-ae02-432ce1ee95ea None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:55.489824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f81ba16f-2bd7-4824-ae02-432ce1ee95ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:55.597512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 194/389] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:55.620317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff82510-db15-4c11-8388-9c3f9a7c7b3b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:55.752808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff82510-db15-4c11-8388-9c3f9a7c7b3b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:55.779141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:55.795918 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4fb46-df77-43ca-a78e-a0837b496a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:55.796976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fb46-df77-43ca-a78e-a0837b496a9e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:55.797445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fb46-df77-43ca-a78e-a0837b496a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:55.797966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4fb46-df77-43ca-a78e-a0837b496a9e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:55.810475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4fb46-df77-43ca-a78e-a0837b496a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-46b4fb46-df77-43ca-a78e-a0837b496a9e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:55.816376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:55.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:55.818148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:55.820772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 196/390] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:11:55.821018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d04d2-2470-4b70-972c-3a25e2da6947 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:55.821485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d04d2-2470-4b70-972c-3a25e2da6947 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:55.826320 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d04d2-2470-4b70-972c-3a25e2da6947 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:55.828257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 195/391] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:11:55.830369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4fb46-df77-43ca-a78e-a0837b496a9e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:55.834118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc25a-da1d-4a32-af65-dbf7ce2adc5e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:55.834644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc25a-da1d-4a32-af65-dbf7ce2adc5e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:55.835128 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704cc25a-da1d-4a32-af65-dbf7ce2adc5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:55.863501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-704cc25a-da1d-4a32-af65-dbf7ce2adc5e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:55.866271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-704cc25a-da1d-4a32-af65-dbf7ce2adc5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:55.980290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4fb46-df77-43ca-a78e-a0837b496a9e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:56.016021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50b215-020d-48d9-a7ea-fc14d7041b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:56.017021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50b215-020d-48d9-a7ea-fc14d7041b0c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:56.017488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50b215-020d-48d9-a7ea-fc14d7041b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:56.017988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef50b215-020d-48d9-a7ea-fc14d7041b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:56.018969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef50b215-020d-48d9-a7ea-fc14d7041b0c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:56.019456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef50b215-020d-48d9-a7ea-fc14d7041b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:56.021097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef50b215-020d-48d9-a7ea-fc14d7041b0c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:56.030943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 197/392] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:56.035409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:56.038558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50b215-020d-48d9-a7ea-fc14d7041b0c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:56.056576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 196/393] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:11:56.089364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:56.092671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc25a-da1d-4a32-af65-dbf7ce2adc5e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:56.158593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef50b215-020d-48d9-a7ea-fc14d7041b0c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:56.196604 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b3050-5213-4931-8fa1-f228a6b2da7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:56.197627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b3050-5213-4931-8fa1-f228a6b2da7b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:56.198150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b3050-5213-4931-8fa1-f228a6b2da7b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:56.198631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b3050-5213-4931-8fa1-f228a6b2da7b None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:56.200058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b3050-5213-4931-8fa1-f228a6b2da7b None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:56.201825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493b3050-5213-4931-8fa1-f228a6b2da7b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:56.204013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 198/394] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:11:56 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:56.209878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b3050-5213-4931-8fa1-f228a6b2da7b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:56.322964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b3050-5213-4931-8fa1-f228a6b2da7b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:56.337529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:56.366762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c14c2-e2f9-4ba1-9919-e8162e3f46ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:56.367830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c14c2-e2f9-4ba1-9919-e8162e3f46ab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:56.368309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c14c2-e2f9-4ba1-9919-e8162e3f46ab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:56.368808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c14c2-e2f9-4ba1-9919-e8162e3f46ab None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:56.370412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c14c2-e2f9-4ba1-9919-e8162e3f46ab None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:56.372129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c14c2-e2f9-4ba1-9919-e8162e3f46ab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:56.374511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 199/395] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:56 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:56.379605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704cc25a-da1d-4a32-af65-dbf7ce2adc5e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:56.380914 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c14c2-e2f9-4ba1-9919-e8162e3f46ab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:56.506058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c14c2-e2f9-4ba1-9919-e8162e3f46ab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:56.543489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93998c-ae9e-4a43-8c4a-03a9564fb193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:56.544561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93998c-ae9e-4a43-8c4a-03a9564fb193 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:56.545053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93998c-ae9e-4a43-8c4a-03a9564fb193 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:56.545512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93998c-ae9e-4a43-8c4a-03a9564fb193 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df/groups/8577807d221144cba4cc471a3d22db2c/roles/5e0c08646046426385776839f1f168cd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:56.548060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93998c-ae9e-4a43-8c4a-03a9564fb193 None admin] RBAC: Authorizing `identity:create_grant(project_id=9ad9190f63064d308a0423e34054b7df, group_id=8577807d221144cba4cc471a3d22db2c, role_id=5e0c08646046426385776839f1f168cd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:56.549772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc93998c-ae9e-4a43-8c4a-03a9564fb193 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:56.565040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 200/396] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Jul 26 07:11:56 2022] PUT /identity/v3/projects/9ad9190f63064d308a0423e34054b7df/groups/8577807d221144cba4cc471a3d22db2c/roles/5e0c08646046426385776839f1f168cd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:56.569847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9671a-464c-4cbe-b667-62fc770779db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:56.569847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9671a-464c-4cbe-b667-62fc770779db None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:56.569847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9671a-464c-4cbe-b667-62fc770779db None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:56.569847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9671a-464c-4cbe-b667-62fc770779db None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:56.607559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9671a-464c-4cbe-b667-62fc770779db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c9f9671a-464c-4cbe-b667-62fc770779db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:56.617300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:56.618281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:56.619059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:56.628158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 197/397] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 536 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:56.634478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93998c-ae9e-4a43-8c4a-03a9564fb193 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:56.686410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:56.763080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93998c-ae9e-4a43-8c4a-03a9564fb193 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:56.804118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e75773-012c-4696-b63f-a650d6237557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:56.805177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e75773-012c-4696-b63f-a650d6237557 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:56.805689 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e75773-012c-4696-b63f-a650d6237557 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:56.806232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e75773-012c-4696-b63f-a650d6237557 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:56.807207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e75773-012c-4696-b63f-a650d6237557 None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:56.807708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e75773-012c-4696-b63f-a650d6237557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:56.809330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e75773-012c-4696-b63f-a650d6237557 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:56.820457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 201/398] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:56.827265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9671a-464c-4cbe-b667-62fc770779db None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:56.866297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00d0e7-8804-43d4-ada2-40048db83d21 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:56.866908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00d0e7-8804-43d4-ada2-40048db83d21 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:56.867683 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00d0e7-8804-43d4-ada2-40048db83d21 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:56.869614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 202/399] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:11:56.875417 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47cd67-913f-4284-aa6b-1d1cbcf3ebcc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:56.876350 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47cd67-913f-4284-aa6b-1d1cbcf3ebcc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:56.876819 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47cd67-913f-4284-aa6b-1d1cbcf3ebcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:56.901647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-de47cd67-913f-4284-aa6b-1d1cbcf3ebcc None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:56.903536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-de47cd67-913f-4284-aa6b-1d1cbcf3ebcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:56.981772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:56.990495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 203/400] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:57.024769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47cd67-913f-4284-aa6b-1d1cbcf3ebcc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:57.059231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9671a-464c-4cbe-b667-62fc770779db None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:57.094151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:57.102847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266106e-a397-4c9d-83d4-ed13d40203e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:57.103531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266106e-a397-4c9d-83d4-ed13d40203e3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:57.103812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266106e-a397-4c9d-83d4-ed13d40203e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:57.104086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266106e-a397-4c9d-83d4-ed13d40203e3 None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:57.105340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266106e-a397-4c9d-83d4-ed13d40203e3 None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:57.106864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8266106e-a397-4c9d-83d4-ed13d40203e3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:57.109532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 198/401] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:11:56 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:57.115766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266106e-a397-4c9d-83d4-ed13d40203e3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:57.220581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47cd67-913f-4284-aa6b-1d1cbcf3ebcc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:57.270810 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65f024-4e37-4745-b215-fd338275090d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:57.272019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65f024-4e37-4745-b215-fd338275090d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:57.272295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65f024-4e37-4745-b215-fd338275090d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:57.272564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65f024-4e37-4745-b215-fd338275090d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:57.286336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65f024-4e37-4745-b215-fd338275090d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1e65f024-4e37-4745-b215-fd338275090d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:57.293182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:57.294119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:11:57.294924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:57.296012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 204/402] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:11:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:57.297229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6493d-6ff7-4238-9bf3-f050431a4b8e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:57.297229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6493d-6ff7-4238-9bf3-f050431a4b8e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:57.297229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b6493d-6ff7-4238-9bf3-f050431a4b8e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:57.299201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 205/403] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:57.302711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6493d-6ff7-4238-9bf3-f050431a4b8e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:57.308349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8266106e-a397-4c9d-83d4-ed13d40203e3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:57.355839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:57.370368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d163bb-499d-404e-aa6d-7a023118b967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:57.371382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d163bb-499d-404e-aa6d-7a023118b967 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:57.371784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d163bb-499d-404e-aa6d-7a023118b967 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:57.372240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d163bb-499d-404e-aa6d-7a023118b967 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:57.374271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d163bb-499d-404e-aa6d-7a023118b967 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:57.377043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d163bb-499d-404e-aa6d-7a023118b967 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:57.380846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 199/404] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:57 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:57.383271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd8c2e-9f75-45fe-a3e1-223d163b5aff None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:57.383871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd8c2e-9f75-45fe-a3e1-223d163b5aff None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:57.384291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd8c2e-9f75-45fe-a3e1-223d163b5aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:57.415592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0dcd8c2e-9f75-45fe-a3e1-223d163b5aff None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:57.417765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0dcd8c2e-9f75-45fe-a3e1-223d163b5aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:57.445422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:57.462360 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b6493d-6ff7-4238-9bf3-f050431a4b8e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:57.515178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122aa1f8-0ed3-4c34-87b6-742fdb098276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:57.516279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122aa1f8-0ed3-4c34-87b6-742fdb098276 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:57.516982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122aa1f8-0ed3-4c34-87b6-742fdb098276 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:57.517451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122aa1f8-0ed3-4c34-87b6-742fdb098276 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:57.519037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122aa1f8-0ed3-4c34-87b6-742fdb098276 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:57.519768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122aa1f8-0ed3-4c34-87b6-742fdb098276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:57.522109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122aa1f8-0ed3-4c34-87b6-742fdb098276 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:57.534879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 206/405] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:57.536618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122aa1f8-0ed3-4c34-87b6-742fdb098276 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:57.553892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 200/406] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:57.555531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd8c2e-9f75-45fe-a3e1-223d163b5aff None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:57.660627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122aa1f8-0ed3-4c34-87b6-742fdb098276 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:57.719490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625c4df-66b0-41ce-b102-ca804d4a877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:57.720812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c4df-66b0-41ce-b102-ca804d4a877f None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:57.721385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c4df-66b0-41ce-b102-ca804d4a877f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:57.721963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625c4df-66b0-41ce-b102-ca804d4a877f None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/groups/8577807d221144cba4cc471a3d22db2c/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:57.725539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625c4df-66b0-41ce-b102-ca804d4a877f None admin] RBAC: Authorizing `identity:create_grant(project_id=e07b3a0db08041fc87412eeb161bcf6f, group_id=8577807d221144cba4cc471a3d22db2c, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:57.728496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625c4df-66b0-41ce-b102-ca804d4a877f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:57.731347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd8c2e-9f75-45fe-a3e1-223d163b5aff None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:57.755677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 207/407] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Jul 26 07:11:57 2022] PUT /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/groups/8577807d221144cba4cc471a3d22db2c/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:11:57.757302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625c4df-66b0-41ce-b102-ca804d4a877f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:57.766571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:57.782672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a41714-0fa6-4777-85f5-7db8eb14f5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:57.783697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a41714-0fa6-4777-85f5-7db8eb14f5fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:57.784193 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a41714-0fa6-4777-85f5-7db8eb14f5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:57.784752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a41714-0fa6-4777-85f5-7db8eb14f5fb None admin] PATH_INFO: `/v3/groups/e2e659b5a94d4f6199595897bbde4c95` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:57.786428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a41714-0fa6-4777-85f5-7db8eb14f5fb None admin] RBAC: Authorizing `identity:get_group(group_id=e2e659b5a94d4f6199595897bbde4c95)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:57.788083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a41714-0fa6-4777-85f5-7db8eb14f5fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:57.789694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:57.791045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 201/408] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:11:57 2022] GET /identity/v3/groups/e2e659b5a94d4f6199595897bbde4c95 => generated 229 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:57.796512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a41714-0fa6-4777-85f5-7db8eb14f5fb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:57.928745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a41714-0fa6-4777-85f5-7db8eb14f5fb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:57.970477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573683d-b4ec-4ad0-96a8-d922130e30b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:57.971606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573683d-b4ec-4ad0-96a8-d922130e30b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:57.972126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573683d-b4ec-4ad0-96a8-d922130e30b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:57.972617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573683d-b4ec-4ad0-96a8-d922130e30b2 None admin] PATH_INFO: `/v3/projects/9aaf3bcccded4ffcb2a432d5c86ce8ea` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:57.974127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573683d-b4ec-4ad0-96a8-d922130e30b2 None admin] RBAC: Authorizing `identity:get_project(project_id=9aaf3bcccded4ffcb2a432d5c86ce8ea)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:57.975822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573683d-b4ec-4ad0-96a8-d922130e30b2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:57.978107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 202/409] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:57 2022] GET /identity/v3/projects/9aaf3bcccded4ffcb2a432d5c86ce8ea => generated 344 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:57.983990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573683d-b4ec-4ad0-96a8-d922130e30b2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:58.013595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625c4df-66b0-41ce-b102-ca804d4a877f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:58.060562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:58.068472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19d855-ed27-41d9-8e59-67a82fbcc2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:58.069528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19d855-ed27-41d9-8e59-67a82fbcc2c0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:58.070255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19d855-ed27-41d9-8e59-67a82fbcc2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:58.070937 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19d855-ed27-41d9-8e59-67a82fbcc2c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:58.086239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19d855-ed27-41d9-8e59-67a82fbcc2c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0e19d855-ed27-41d9-8e59-67a82fbcc2c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:58.092590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:58.093491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:58.094410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 208/410] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:58.107720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19d855-ed27-41d9-8e59-67a82fbcc2c0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:58.201044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573683d-b4ec-4ad0-96a8-d922130e30b2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:58.210390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:58.264152 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19d855-ed27-41d9-8e59-67a82fbcc2c0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:58.293927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620401a9-1be4-4e66-9ae0-a706dc9b23e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:58.296642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620401a9-1be4-4e66-9ae0-a706dc9b23e4 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:58.296929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620401a9-1be4-4e66-9ae0-a706dc9b23e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:58.297197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620401a9-1be4-4e66-9ae0-a706dc9b23e4 None admin] PATH_INFO: `/v3/projects/9aaf3bcccded4ffcb2a432d5c86ce8ea/groups/e2e659b5a94d4f6199595897bbde4c95/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:58.302081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620401a9-1be4-4e66-9ae0-a706dc9b23e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9aaf3bcccded4ffcb2a432d5c86ce8ea, group_id=e2e659b5a94d4f6199595897bbde4c95, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:58.303590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620401a9-1be4-4e66-9ae0-a706dc9b23e4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:58.335317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 203/411] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Jul 26 07:11:57 2022] PUT /identity/v3/projects/9aaf3bcccded4ffcb2a432d5c86ce8ea/groups/e2e659b5a94d4f6199595897bbde4c95/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:11:58.338635 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ae92-0dba-46ba-99d8-f47dfe8deca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:58.339308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ae92-0dba-46ba-99d8-f47dfe8deca1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:58.339596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ae92-0dba-46ba-99d8-f47dfe8deca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:58.339873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e900ae92-0dba-46ba-99d8-f47dfe8deca1 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:58.340696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ae92-0dba-46ba-99d8-f47dfe8deca1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:58.341055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ae92-0dba-46ba-99d8-f47dfe8deca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:58.342569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e900ae92-0dba-46ba-99d8-f47dfe8deca1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:58.353432 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 209/412] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:11:58.361930 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620401a9-1be4-4e66-9ae0-a706dc9b23e4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:58.496747 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620401a9-1be4-4e66-9ae0-a706dc9b23e4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:58.520059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:58.535674 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9fd3b-92ff-4846-af1c-0990b3181710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:58.537071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9fd3b-92ff-4846-af1c-0990b3181710 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:58.537466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9fd3b-92ff-4846-af1c-0990b3181710 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:58.537854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9fd3b-92ff-4846-af1c-0990b3181710 None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:58.539479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9fd3b-92ff-4846-af1c-0990b3181710 None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:58.541092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e9fd3b-92ff-4846-af1c-0990b3181710 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:58.543420 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 204/413] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:11:58 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:58.551676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ae92-0dba-46ba-99d8-f47dfe8deca1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:58.638423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb6d99-03a7-4e94-80bc-88c27584ea0f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:58.638809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb6d99-03a7-4e94-80bc-88c27584ea0f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:58.639273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb6d99-03a7-4e94-80bc-88c27584ea0f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:58.647870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 205/414] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:11:58.650083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc835bf0-717b-4793-87d9-7690a7820bdb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:58.650545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc835bf0-717b-4793-87d9-7690a7820bdb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:58.650997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc835bf0-717b-4793-87d9-7690a7820bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:58.667713 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e900ae92-0dba-46ba-99d8-f47dfe8deca1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:58.708196 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7baff-ec9c-4d3e-b38f-9a9b57240238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:58.709114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7baff-ec9c-4d3e-b38f-9a9b57240238 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:58.709516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7baff-ec9c-4d3e-b38f-9a9b57240238 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:58.709933 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad7baff-ec9c-4d3e-b38f-9a9b57240238 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:58.712364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7baff-ec9c-4d3e-b38f-9a9b57240238 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:58.714341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad7baff-ec9c-4d3e-b38f-9a9b57240238 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:58.716835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 210/415] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:11:58 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:58.727517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fc835bf0-717b-4793-87d9-7690a7820bdb None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:58.729507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fc835bf0-717b-4793-87d9-7690a7820bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:58.736887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7baff-ec9c-4d3e-b38f-9a9b57240238 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:58.771157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:58.863760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad7baff-ec9c-4d3e-b38f-9a9b57240238 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:58.874676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 206/416] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:58.913639 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc835bf0-717b-4793-87d9-7690a7820bdb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:58.930527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fb5d6-da7d-45d9-91da-490ebdfb039a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:58.931408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fb5d6-da7d-45d9-91da-490ebdfb039a None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:58.931807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fb5d6-da7d-45d9-91da-490ebdfb039a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:58.932185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fb5d6-da7d-45d9-91da-490ebdfb039a None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/groups/8577807d221144cba4cc471a3d22db2c/roles/5e0c08646046426385776839f1f168cd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:58.934741 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fb5d6-da7d-45d9-91da-490ebdfb039a None admin] RBAC: Authorizing `identity:create_grant(project_id=e07b3a0db08041fc87412eeb161bcf6f, group_id=8577807d221144cba4cc471a3d22db2c, role_id=5e0c08646046426385776839f1f168cd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:58.936336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fb5d6-da7d-45d9-91da-490ebdfb039a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:58.962807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 211/417] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Jul 26 07:11:58 2022] PUT /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f/groups/8577807d221144cba4cc471a3d22db2c/roles/5e0c08646046426385776839f1f168cd => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:11:59.109386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc835bf0-717b-4793-87d9-7690a7820bdb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:59.137269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:59.142869 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97313c46-da80-4b3f-b62a-dfc2d19cfb8c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:59.144483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97313c46-da80-4b3f-b62a-dfc2d19cfb8c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:59.144953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97313c46-da80-4b3f-b62a-dfc2d19cfb8c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:59.147684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 212/418] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:11:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:11:59.153878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c1456-324b-442c-8e07-6607f9b5c55c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:59.154580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c1456-324b-442c-8e07-6607f9b5c55c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:59.155047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7c1456-324b-442c-8e07-6607f9b5c55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:59.175915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699da5a3-dd27-4d29-b30e-91c94f7d70ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:59.176796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699da5a3-dd27-4d29-b30e-91c94f7d70ec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:59.177566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699da5a3-dd27-4d29-b30e-91c94f7d70ec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:59.178044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699da5a3-dd27-4d29-b30e-91c94f7d70ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:59.192197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1a7c1456-324b-442c-8e07-6607f9b5c55c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:11:59.194191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1a7c1456-324b-442c-8e07-6607f9b5c55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:11:59.201767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699da5a3-dd27-4d29-b30e-91c94f7d70ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-699da5a3-dd27-4d29-b30e-91c94f7d70ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:59.213780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:59.214724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:11:59.215499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:11:59.217887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 207/419] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:11:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:59.225325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699da5a3-dd27-4d29-b30e-91c94f7d70ec None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:59.335536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:59.349978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 213/420] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:11:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:11:59.374754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c1456-324b-442c-8e07-6607f9b5c55c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:59.426857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699da5a3-dd27-4d29-b30e-91c94f7d70ec None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:59.469274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d4c0f-10c9-4a05-9cfa-23ccd13c43eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:59.470089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4c0f-10c9-4a05-9cfa-23ccd13c43eb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:59.470702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4c0f-10c9-4a05-9cfa-23ccd13c43eb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:59.471071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4c0f-10c9-4a05-9cfa-23ccd13c43eb None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:59.472070 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d4c0f-10c9-4a05-9cfa-23ccd13c43eb None admin] RBAC: Adding query filter params (name=member) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:11:59.472471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d4c0f-10c9-4a05-9cfa-23ccd13c43eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:59.474029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d4c0f-10c9-4a05-9cfa-23ccd13c43eb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:59.484368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 208/421] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:11:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:11:59.490286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d4c0f-10c9-4a05-9cfa-23ccd13c43eb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:59.572301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:59.598286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7c1456-324b-442c-8e07-6607f9b5c55c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:59.671679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248b7da-9d17-439a-b8e8-bcd36753b1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:59.672345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248b7da-9d17-439a-b8e8-bcd36753b1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:59.672661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248b7da-9d17-439a-b8e8-bcd36753b1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:59.672934 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248b7da-9d17-439a-b8e8-bcd36753b1b9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:59.688882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d4c0f-10c9-4a05-9cfa-23ccd13c43eb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:59.694654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248b7da-9d17-439a-b8e8-bcd36753b1b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0248b7da-9d17-439a-b8e8-bcd36753b1b9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:11:59.702509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:11:59.703485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:11:59.704283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 214/422] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:11:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:11:59.712719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248b7da-9d17-439a-b8e8-bcd36753b1b9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:11:59.748542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf671d4-286c-423d-b3ce-f45a2227929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:11:59.749191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf671d4-286c-423d-b3ce-f45a2227929a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:11:59.749470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf671d4-286c-423d-b3ce-f45a2227929a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:11:59.749758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf671d4-286c-423d-b3ce-f45a2227929a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:11:59.797375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:11:59.906007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248b7da-9d17-439a-b8e8-bcd36753b1b9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:11:59.906871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf671d4-286c-423d-b3ce-f45a2227929a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:11:59.908359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf671d4-286c-423d-b3ce-f45a2227929a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:11:59.964248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:00.006866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18481c2-ac52-4680-a6ec-f32ea7553cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:00.007963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18481c2-ac52-4680-a6ec-f32ea7553cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:00.008472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18481c2-ac52-4680-a6ec-f32ea7553cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:00.008968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18481c2-ac52-4680-a6ec-f32ea7553cc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:00.010379 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18481c2-ac52-4680-a6ec-f32ea7553cc6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:00.010938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18481c2-ac52-4680-a6ec-f32ea7553cc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:00.013661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18481c2-ac52-4680-a6ec-f32ea7553cc6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:00.048975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 215/423] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:11:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:00.067679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18481c2-ac52-4680-a6ec-f32ea7553cc6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:00.131115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 209/424] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:11:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 641 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 07:12:00.137172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf671d4-286c-423d-b3ce-f45a2227929a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:00.224292 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18481c2-ac52-4680-a6ec-f32ea7553cc6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:00.241113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:00.268094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5945f-e92e-4fad-ae3c-060b404195c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:00.269278 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5945f-e92e-4fad-ae3c-060b404195c6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:00.269779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5945f-e92e-4fad-ae3c-060b404195c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:00.270341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5945f-e92e-4fad-ae3c-060b404195c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:00.309247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf671d4-286c-423d-b3ce-f45a2227929a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:00.344668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5945f-e92e-4fad-ae3c-060b404195c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:00.346665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5945f-e92e-4fad-ae3c-060b404195c6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:00.369397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e78c12-6e46-4db0-a1a6-ec518430191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:00.370685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e78c12-6e46-4db0-a1a6-ec518430191a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:00.371043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e78c12-6e46-4db0-a1a6-ec518430191a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:00.371542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e78c12-6e46-4db0-a1a6-ec518430191a None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:00.373576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e78c12-6e46-4db0-a1a6-ec518430191a None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:00.376377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e78c12-6e46-4db0-a1a6-ec518430191a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:00.384431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 210/425] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:12:00 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:00.384431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4308ab-a61e-4c9a-b7b6-1496221e9b93 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:00.384431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4308ab-a61e-4c9a-b7b6-1496221e9b93 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:00.384431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4308ab-a61e-4c9a-b7b6-1496221e9b93 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:00.385941 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 211/426] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:00.387240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4308ab-a61e-4c9a-b7b6-1496221e9b93 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:00.406572 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:00.477772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 216/427] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 07:12:00.479484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084ab6b-d9c9-44f6-be22-a6e26cb8126a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:00.479907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084ab6b-d9c9-44f6-be22-a6e26cb8126a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:00.480336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084ab6b-d9c9-44f6-be22-a6e26cb8126a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:00.499827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f084ab6b-d9c9-44f6-be22-a6e26cb8126a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:00.501527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f084ab6b-d9c9-44f6-be22-a6e26cb8126a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:00.513211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:00.576029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4308ab-a61e-4c9a-b7b6-1496221e9b93 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:00.578671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 217/428] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:00.579788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084ab6b-d9c9-44f6-be22-a6e26cb8126a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:00.639668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739d608-63c6-4b70-aa08-db7ce5751ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:00.640976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739d608-63c6-4b70-aa08-db7ce5751ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:00.641604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739d608-63c6-4b70-aa08-db7ce5751ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:00.642468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739d608-63c6-4b70-aa08-db7ce5751ba3 None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:00.644492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739d608-63c6-4b70-aa08-db7ce5751ba3 None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:00.647453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739d608-63c6-4b70-aa08-db7ce5751ba3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:00.650539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 212/429] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:12:00 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:00.652027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739d608-63c6-4b70-aa08-db7ce5751ba3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:00.692261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084ab6b-d9c9-44f6-be22-a6e26cb8126a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:00.730928 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f06d9b-fc7e-4412-bb1c-5398d9685a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:00.731945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f06d9b-fc7e-4412-bb1c-5398d9685a39 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:00.732402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f06d9b-fc7e-4412-bb1c-5398d9685a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:00.732824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f06d9b-fc7e-4412-bb1c-5398d9685a39 None admin] PATH_INFO: `/v3/projects/9ad9190f63064d308a0423e34054b7df` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:00.734603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f06d9b-fc7e-4412-bb1c-5398d9685a39 None admin] RBAC: Authorizing `identity:get_project(project_id=9ad9190f63064d308a0423e34054b7df)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:00.736299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f06d9b-fc7e-4412-bb1c-5398d9685a39 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:00.738597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 218/430] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:12:00 2022] GET /identity/v3/projects/9ad9190f63064d308a0423e34054b7df => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:00.739592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f06d9b-fc7e-4412-bb1c-5398d9685a39 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:00.757138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:00.795596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:00.833654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739d608-63c6-4b70-aa08-db7ce5751ba3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:00.856080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f06d9b-fc7e-4412-bb1c-5398d9685a39 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:00.887699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658972a3-70fa-4411-a881-bc56cb766b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:00.888554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658972a3-70fa-4411-a881-bc56cb766b42 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:00.888933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658972a3-70fa-4411-a881-bc56cb766b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:00.889306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658972a3-70fa-4411-a881-bc56cb766b42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:00.897034 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208f531-533e-4eb9-8751-d4aae789684c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:00.897969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208f531-533e-4eb9-8751-d4aae789684c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:00.898383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208f531-533e-4eb9-8751-d4aae789684c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:00.898812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208f531-533e-4eb9-8751-d4aae789684c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:00.899762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208f531-533e-4eb9-8751-d4aae789684c None admin] RBAC: Adding query filter params (group.id=8577807d221144cba4cc471a3d22db2c, role.id=64dfc66a8abe4468ac4f1ca071d28bd4, scope.project.id=9ad9190f63064d308a0423e34054b7df) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:00.900265 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208f531-533e-4eb9-8751-d4aae789684c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:00.900953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658972a3-70fa-4411-a881-bc56cb766b42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:00.902154 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c208f531-533e-4eb9-8751-d4aae789684c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-658972a3-70fa-4411-a881-bc56cb766b42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:00.909032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:00.910031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:00.910818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 213/432] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:00.911804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 219/432] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jul 26 07:12:00 2022] GET /identity/v3/role_assignments?group.id=8577807d221144cba4cc471a3d22db2c&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:00.913918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658972a3-70fa-4411-a881-bc56cb766b42 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:00.921548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208f531-533e-4eb9-8751-d4aae789684c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:01.084330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:01.112807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c208f531-533e-4eb9-8751-d4aae789684c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:01.115912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658972a3-70fa-4411-a881-bc56cb766b42 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:01.153948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21e94f-44ab-4999-b480-420fa3d9830f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:01.154592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21e94f-44ab-4999-b480-420fa3d9830f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:01.154873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21e94f-44ab-4999-b480-420fa3d9830f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:01.155137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21e94f-44ab-4999-b480-420fa3d9830f None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:01.155877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21e94f-44ab-4999-b480-420fa3d9830f None admin] RBAC: Adding query filter params (name=member) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:01.156181 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21e94f-44ab-4999-b480-420fa3d9830f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:01.157595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21e94f-44ab-4999-b480-420fa3d9830f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:01.168669 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 220/433] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:12:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:01.173642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21e94f-44ab-4999-b480-420fa3d9830f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:01.186571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6170a2-9bf2-449b-bee8-ea763a3de9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:01.188628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6170a2-9bf2-449b-bee8-ea763a3de9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:01.189367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6170a2-9bf2-449b-bee8-ea763a3de9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:01.190188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6170a2-9bf2-449b-bee8-ea763a3de9e4 None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:01.193053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6170a2-9bf2-449b-bee8-ea763a3de9e4 None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:01.197142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6170a2-9bf2-449b-bee8-ea763a3de9e4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:01.205880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 214/434] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:12:00 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:01.211428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6170a2-9bf2-449b-bee8-ea763a3de9e4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:01.218181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:01.337442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21e94f-44ab-4999-b480-420fa3d9830f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:01.353460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6170a2-9bf2-449b-bee8-ea763a3de9e4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:01.383691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26810ed8-f2e3-4d4b-a5c2-aba9be74bec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:01.384340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26810ed8-f2e3-4d4b-a5c2-aba9be74bec7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:01.384623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26810ed8-f2e3-4d4b-a5c2-aba9be74bec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:01.384899 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26810ed8-f2e3-4d4b-a5c2-aba9be74bec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:01.394776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:01.414899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86faac3-c11f-417b-96d4-82e44f36b6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:01.415547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86faac3-c11f-417b-96d4-82e44f36b6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:01.415858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86faac3-c11f-417b-96d4-82e44f36b6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:01.416088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86faac3-c11f-417b-96d4-82e44f36b6c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:01.416893 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86faac3-c11f-417b-96d4-82e44f36b6c0 None admin] RBAC: Adding query filter params (group.id=8577807d221144cba4cc471a3d22db2c, role.id=5e0c08646046426385776839f1f168cd, scope.project.id=9ad9190f63064d308a0423e34054b7df) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:01.417210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86faac3-c11f-417b-96d4-82e44f36b6c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:01.419343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86faac3-c11f-417b-96d4-82e44f36b6c0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:01.429665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 215/435] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jul 26 07:12:01 2022] GET /identity/v3/role_assignments?group.id=8577807d221144cba4cc471a3d22db2c&scope.project.id=9ad9190f63064d308a0423e34054b7df&role.id=5e0c08646046426385776839f1f168cd => generated 611 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:01.431111 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412e488-16b3-45e3-a0fa-765bbe6b3561 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:01.431496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412e488-16b3-45e3-a0fa-765bbe6b3561 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:01.432609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8412e488-16b3-45e3-a0fa-765bbe6b3561 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:01.433298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 216/436] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:01.439242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99bfc1-f083-4b3a-8ce4-7ed156008e7c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:01.439580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99bfc1-f083-4b3a-8ce4-7ed156008e7c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:01.439814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99bfc1-f083-4b3a-8ce4-7ed156008e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:01.463830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6b99bfc1-f083-4b3a-8ce4-7ed156008e7c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:01.465045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6b99bfc1-f083-4b3a-8ce4-7ed156008e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:01.479449 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26810ed8-f2e3-4d4b-a5c2-aba9be74bec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:01.481273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26810ed8-f2e3-4d4b-a5c2-aba9be74bec7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:01.553389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:01.560582 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 217/437] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:12:01.586811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99bfc1-f083-4b3a-8ce4-7ed156008e7c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:01.632507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 221/438] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 26 07:12:01.638353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26810ed8-f2e3-4d4b-a5c2-aba9be74bec7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:01.686472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:01.716959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99bfc1-f083-4b3a-8ce4-7ed156008e7c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:01.757837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c934786-8638-42b6-a552-346f83b923f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:01.757837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c934786-8638-42b6-a552-346f83b923f6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:01.757837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c934786-8638-42b6-a552-346f83b923f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:01.757837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c934786-8638-42b6-a552-346f83b923f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:01.771378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c934786-8638-42b6-a552-346f83b923f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:01.775705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26810ed8-f2e3-4d4b-a5c2-aba9be74bec7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1c934786-8638-42b6-a552-346f83b923f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:01.776559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:01.777426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:01.778477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 218/439] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:01.779495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecdc5f-c6ab-4bed-9764-1253b5225b8a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:01.779610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecdc5f-c6ab-4bed-9764-1253b5225b8a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:01.779887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecdc5f-c6ab-4bed-9764-1253b5225b8a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:01.781968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 219/440] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:01.788140 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecdc5f-c6ab-4bed-9764-1253b5225b8a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:01.843145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6943cf-8f9c-444b-9100-21991c0420f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:01.844199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6943cf-8f9c-444b-9100-21991c0420f4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:01.844668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6943cf-8f9c-444b-9100-21991c0420f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:01.845115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6943cf-8f9c-444b-9100-21991c0420f4 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:01.847294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6943cf-8f9c-444b-9100-21991c0420f4 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:01.849763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6943cf-8f9c-444b-9100-21991c0420f4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:01.855444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 222/441] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:12:01 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:01.855994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94eb5f1-90a4-4697-b822-a18ceccb88d2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:01.856455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94eb5f1-90a4-4697-b822-a18ceccb88d2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:01.856976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94eb5f1-90a4-4697-b822-a18ceccb88d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:01.884959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f94eb5f1-90a4-4697-b822-a18ceccb88d2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:01.886691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f94eb5f1-90a4-4697-b822-a18ceccb88d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:01.890102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:01.963460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecdc5f-c6ab-4bed-9764-1253b5225b8a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:01.997787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 223/442] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:01.998932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94eb5f1-90a4-4697-b822-a18ceccb88d2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:02.008358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f769b6-3f22-48a4-9042-76ad79b915b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:02.009194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f769b6-3f22-48a4-9042-76ad79b915b4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:02.009690 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f769b6-3f22-48a4-9042-76ad79b915b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:02.010151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f769b6-3f22-48a4-9042-76ad79b915b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:02.011023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f769b6-3f22-48a4-9042-76ad79b915b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:02.011565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f769b6-3f22-48a4-9042-76ad79b915b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:02.013235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f769b6-3f22-48a4-9042-76ad79b915b4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:02.024553 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 220/443] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:02.026085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f769b6-3f22-48a4-9042-76ad79b915b4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:02.038102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:02.118522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94eb5f1-90a4-4697-b822-a18ceccb88d2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:02.135657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f769b6-3f22-48a4-9042-76ad79b915b4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:02.154058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc6013-657b-458f-95e6-5613f2bbf755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:02.154711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc6013-657b-458f-95e6-5613f2bbf755 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:02.154987 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc6013-657b-458f-95e6-5613f2bbf755 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:02.155257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dc6013-657b-458f-95e6-5613f2bbf755 None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:02.155848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:02.156407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc6013-657b-458f-95e6-5613f2bbf755 None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:02.157991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dc6013-657b-458f-95e6-5613f2bbf755 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:02.159884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 224/444] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:12:01 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:02.160676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc6013-657b-458f-95e6-5613f2bbf755 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:02.177008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89778-52bc-4e42-9142-23b5e5361628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:02.177664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89778-52bc-4e42-9142-23b5e5361628 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:02.177968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89778-52bc-4e42-9142-23b5e5361628 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:02.178251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba89778-52bc-4e42-9142-23b5e5361628 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:02.189775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba89778-52bc-4e42-9142-23b5e5361628 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8ba89778-52bc-4e42-9142-23b5e5361628 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:02.195521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:02.197253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 221/445] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:02.198777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89778-52bc-4e42-9142-23b5e5361628 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:02.269287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dc6013-657b-458f-95e6-5613f2bbf755 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:02.271780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:02.299699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba89778-52bc-4e42-9142-23b5e5361628 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:02.308025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead518a8-5007-4f5b-9a04-c186336296b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:02.308673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead518a8-5007-4f5b-9a04-c186336296b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:02.308952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead518a8-5007-4f5b-9a04-c186336296b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:02.309215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead518a8-5007-4f5b-9a04-c186336296b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:02.334461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038acd46-1dfd-4ac6-af77-c730e3361ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:02.335174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038acd46-1dfd-4ac6-af77-c730e3361ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:02.335536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038acd46-1dfd-4ac6-af77-c730e3361ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:02.335862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038acd46-1dfd-4ac6-af77-c730e3361ab5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:02.337129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038acd46-1dfd-4ac6-af77-c730e3361ab5 None admin] RBAC: Adding query filter params (group.id=8577807d221144cba4cc471a3d22db2c, role.id=64dfc66a8abe4468ac4f1ca071d28bd4, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:02.337564 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038acd46-1dfd-4ac6-af77-c730e3361ab5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:02.339442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038acd46-1dfd-4ac6-af77-c730e3361ab5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:02.345926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 222/446] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jul 26 07:12:02 2022] GET /identity/v3/role_assignments?group.id=8577807d221144cba4cc471a3d22db2c&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=64dfc66a8abe4468ac4f1ca071d28bd4 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:02.346796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038acd46-1dfd-4ac6-af77-c730e3361ab5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:02.394176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead518a8-5007-4f5b-9a04-c186336296b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:02.396408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead518a8-5007-4f5b-9a04-c186336296b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:02.419176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:02.487187 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038acd46-1dfd-4ac6-af77-c730e3361ab5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:02.527152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e73c90-6e41-460a-aef4-43ea090b76d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:02.528538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e73c90-6e41-460a-aef4-43ea090b76d7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:02.528538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e73c90-6e41-460a-aef4-43ea090b76d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:02.528538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e73c90-6e41-460a-aef4-43ea090b76d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:02.528925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 225/447] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:02.529329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e73c90-6e41-460a-aef4-43ea090b76d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:02.529596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e73c90-6e41-460a-aef4-43ea090b76d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:02.529856 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead518a8-5007-4f5b-9a04-c186336296b6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:02.531045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e73c90-6e41-460a-aef4-43ea090b76d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:02.534174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:02.541090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 223/448] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:02.547649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e73c90-6e41-460a-aef4-43ea090b76d7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:02.664593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead518a8-5007-4f5b-9a04-c186336296b6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:02.665066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e73c90-6e41-460a-aef4-43ea090b76d7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:02.673060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:02.710383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb20935-074d-4e90-b7d8-96d596845ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:02.711144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0200a7-ac47-49d0-8f25-9b9d142fe0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:02.711285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb20935-074d-4e90-b7d8-96d596845ebc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:02.711558 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb20935-074d-4e90-b7d8-96d596845ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:02.711825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0200a7-ac47-49d0-8f25-9b9d142fe0eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:02.711920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb20935-074d-4e90-b7d8-96d596845ebc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:02.712106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0200a7-ac47-49d0-8f25-9b9d142fe0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:02.712397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0200a7-ac47-49d0-8f25-9b9d142fe0eb None admin] PATH_INFO: `/v3/projects/9aaf3bcccded4ffcb2a432d5c86ce8ea` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:02.713703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0200a7-ac47-49d0-8f25-9b9d142fe0eb None admin] RBAC: Authorizing `identity:get_project(project_id=9aaf3bcccded4ffcb2a432d5c86ce8ea)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:02.717857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0200a7-ac47-49d0-8f25-9b9d142fe0eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:02.717857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 226/449] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:12:02 2022] GET /identity/v3/projects/9aaf3bcccded4ffcb2a432d5c86ce8ea => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:02.723157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0200a7-ac47-49d0-8f25-9b9d142fe0eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:02.788670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb20935-074d-4e90-b7d8-96d596845ebc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:02.790225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb20935-074d-4e90-b7d8-96d596845ebc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:02.804087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:02.836883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0200a7-ac47-49d0-8f25-9b9d142fe0eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:02.874766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d996b3-5a3f-4282-8e6a-68c5490fc9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:02.875433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d996b3-5a3f-4282-8e6a-68c5490fc9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:02.875707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d996b3-5a3f-4282-8e6a-68c5490fc9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:02.875981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d996b3-5a3f-4282-8e6a-68c5490fc9b6 None admin] PATH_INFO: `/v3/groups/e2e659b5a94d4f6199595897bbde4c95` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:02.877183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d996b3-5a3f-4282-8e6a-68c5490fc9b6 None admin] RBAC: Authorizing `identity:get_group(group_id=e2e659b5a94d4f6199595897bbde4c95)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:02.878758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d996b3-5a3f-4282-8e6a-68c5490fc9b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:02.880386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 227/450] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:12:02 2022] GET /identity/v3/groups/e2e659b5a94d4f6199595897bbde4c95 => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:02.886441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d996b3-5a3f-4282-8e6a-68c5490fc9b6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:02.902894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 224/451] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:02.908723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb20935-074d-4e90-b7d8-96d596845ebc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:02.917448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:02.991878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d996b3-5a3f-4282-8e6a-68c5490fc9b6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:03.015924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb20935-074d-4e90-b7d8-96d596845ebc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:03.037289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db92c6-3ccd-4595-8076-cb517d332ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:03.038293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db92c6-3ccd-4595-8076-cb517d332ddf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:03.038752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db92c6-3ccd-4595-8076-cb517d332ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:03.039195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61db92c6-3ccd-4595-8076-cb517d332ddf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:03.040399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db92c6-3ccd-4595-8076-cb517d332ddf None admin] RBAC: Adding query filter params (group.id=e2e659b5a94d4f6199595897bbde4c95, role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=9aaf3bcccded4ffcb2a432d5c86ce8ea) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:03.040894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db92c6-3ccd-4595-8076-cb517d332ddf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:03.043407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61db92c6-3ccd-4595-8076-cb517d332ddf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:03.051747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 228/452] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jul 26 07:12:02 2022] GET /identity/v3/role_assignments?group.id=e2e659b5a94d4f6199595897bbde4c95&scope.project.id=9aaf3bcccded4ffcb2a432d5c86ce8ea&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 611 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:03.074034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60902f-cda0-44cf-99c8-4b136bee91a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:03.074844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60902f-cda0-44cf-99c8-4b136bee91a0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:03.075202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60902f-cda0-44cf-99c8-4b136bee91a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:03.075569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd60902f-cda0-44cf-99c8-4b136bee91a0 None admin] PATH_INFO: `/v3/projects/e07b3a0db08041fc87412eeb161bcf6f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:03.077111 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60902f-cda0-44cf-99c8-4b136bee91a0 None admin] RBAC: Authorizing `identity:get_project(project_id=e07b3a0db08041fc87412eeb161bcf6f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:03.078841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd60902f-cda0-44cf-99c8-4b136bee91a0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:03.085815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 225/453] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:12:02 2022] GET /identity/v3/projects/e07b3a0db08041fc87412eeb161bcf6f => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:03.087016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db92c6-3ccd-4595-8076-cb517d332ddf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:03.091495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:03.208389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61db92c6-3ccd-4595-8076-cb517d332ddf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:03.248219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5526676-bf23-4a14-912d-8875bc1db82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:03.248883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5526676-bf23-4a14-912d-8875bc1db82f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:03.249165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5526676-bf23-4a14-912d-8875bc1db82f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:03.249446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5526676-bf23-4a14-912d-8875bc1db82f None admin] PATH_INFO: `/v3/groups/8577807d221144cba4cc471a3d22db2c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:03.250692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5526676-bf23-4a14-912d-8875bc1db82f None admin] RBAC: Authorizing `identity:get_group(group_id=8577807d221144cba4cc471a3d22db2c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:03.252146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5526676-bf23-4a14-912d-8875bc1db82f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:03.253852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 229/454] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Jul 26 07:12:03 2022] GET /identity/v3/groups/8577807d221144cba4cc471a3d22db2c => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:03.261689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60902f-cda0-44cf-99c8-4b136bee91a0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:03.344778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:03.372287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd60902f-cda0-44cf-99c8-4b136bee91a0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:03.407893 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eeac95-81ad-48ed-a875-60c063010969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:03.408518 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeac95-81ad-48ed-a875-60c063010969 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:03.408794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeac95-81ad-48ed-a875-60c063010969 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:03.409068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eeac95-81ad-48ed-a875-60c063010969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:03.409940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeac95-81ad-48ed-a875-60c063010969 None admin] RBAC: Adding query filter params (group.id=8577807d221144cba4cc471a3d22db2c, role.id=5e0c08646046426385776839f1f168cd, scope.project.id=e07b3a0db08041fc87412eeb161bcf6f) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:03.410248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeac95-81ad-48ed-a875-60c063010969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:03.412224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eeac95-81ad-48ed-a875-60c063010969 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:03.418682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 226/455] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Jul 26 07:12:03 2022] GET /identity/v3/role_assignments?group.id=8577807d221144cba4cc471a3d22db2c&scope.project.id=e07b3a0db08041fc87412eeb161bcf6f&role.id=5e0c08646046426385776839f1f168cd => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:05.903626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a8529-8146-417e-805a-ca2be25a8e7d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:05.904823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a8529-8146-417e-805a-ca2be25a8e7d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:05.905283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a8529-8146-417e-805a-ca2be25a8e7d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:05.907265 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 230/456] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:05.912718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10288b-80d9-43c3-86db-9f7e8d02b2cc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:05.913639 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10288b-80d9-43c3-86db-9f7e8d02b2cc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:05.914149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10288b-80d9-43c3-86db-9f7e8d02b2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:05.942876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ff10288b-80d9-43c3-86db-9f7e8d02b2cc None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:05.944499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff10288b-80d9-43c3-86db-9f7e8d02b2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:06.040016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 227/457] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:06.080205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a8529-8146-417e-805a-ca2be25a8e7d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:06.240224 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:06.286489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a8529-8146-417e-805a-ca2be25a8e7d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:06.334588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab73c4-3551-4981-85ac-f4cc7a1279c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:06.335504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab73c4-3551-4981-85ac-f4cc7a1279c6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:06.335712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab73c4-3551-4981-85ac-f4cc7a1279c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:06.335971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab73c4-3551-4981-85ac-f4cc7a1279c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:06.350900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab73c4-3551-4981-85ac-f4cc7a1279c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:06.352425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab73c4-3551-4981-85ac-f4cc7a1279c6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-14ab73c4-3551-4981-85ac-f4cc7a1279c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:06.374022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:06.374949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:06.375781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:06.375781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:06.375781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:06.375781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:06.375781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 231/458] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:06.381173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10288b-80d9-43c3-86db-9f7e8d02b2cc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:06.527438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:06.576723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10288b-80d9-43c3-86db-9f7e8d02b2cc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:06.624735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e18b6d-71e3-4acd-a76e-ab11ba3e7a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:06.625397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e18b6d-71e3-4acd-a76e-ab11ba3e7a8a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:06.625686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e18b6d-71e3-4acd-a76e-ab11ba3e7a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:06.625998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e18b6d-71e3-4acd-a76e-ab11ba3e7a8a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:06.626729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e18b6d-71e3-4acd-a76e-ab11ba3e7a8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:06.627077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e18b6d-71e3-4acd-a76e-ab11ba3e7a8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:06.628681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e18b6d-71e3-4acd-a76e-ab11ba3e7a8a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:06.653932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 228/459] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:06.659321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab73c4-3551-4981-85ac-f4cc7a1279c6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:06.684037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c0fe9-3ffc-4758-ab0d-5abe6322c072 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:06.684556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c0fe9-3ffc-4758-ab0d-5abe6322c072 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:06.684970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c0fe9-3ffc-4758-ab0d-5abe6322c072 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:06.686771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 229/460] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:12:06.692251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f6d1-481e-4ed2-95f0-068c15994ca1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:06.692761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f6d1-481e-4ed2-95f0-068c15994ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:06.693172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474f6d1-481e-4ed2-95f0-068c15994ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:06.723020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3474f6d1-481e-4ed2-95f0-068c15994ca1 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:06.725124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3474f6d1-481e-4ed2-95f0-068c15994ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:06.824955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab73c4-3551-4981-85ac-f4cc7a1279c6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:06.829593 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 230/461] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:12:06.834715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeaf22-adac-43cb-a425-3849247d773e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:06.835286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeaf22-adac-43cb-a425-3849247d773e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:06.835756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eeaf22-adac-43cb-a425-3849247d773e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:06.843787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 231/462] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:06.847555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debecda4-e90e-4371-b5e4-3524a71680ce None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:06.848229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debecda4-e90e-4371-b5e4-3524a71680ce None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:06.850081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debecda4-e90e-4371-b5e4-3524a71680ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:06.869559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:06.880104 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-debecda4-e90e-4371-b5e4-3524a71680ce None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:06.881297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-debecda4-e90e-4371-b5e4-3524a71680ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:06.884046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20ca10-2583-4f9c-a7b6-7ecbc6a9e6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:06.885395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20ca10-2583-4f9c-a7b6-7ecbc6a9e6ea None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:06.886025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20ca10-2583-4f9c-a7b6-7ecbc6a9e6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:06.886492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20ca10-2583-4f9c-a7b6-7ecbc6a9e6ea None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:06.887446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20ca10-2583-4f9c-a7b6-7ecbc6a9e6ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:06.889175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e20ca10-2583-4f9c-a7b6-7ecbc6a9e6ea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:06.918520 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 232/463] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:12:06 2022] POST /identity/v3/users => generated 246 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:06.920397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20ca10-2583-4f9c-a7b6-7ecbc6a9e6ea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:06.984157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:07.025586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 232/464] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:07.067806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debecda4-e90e-4371-b5e4-3524a71680ce None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:07.170146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20ca10-2583-4f9c-a7b6-7ecbc6a9e6ea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:07.219473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f75a45-fee6-4366-b814-52fa16d24f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:07.220205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f75a45-fee6-4366-b814-52fa16d24f77 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:07.220488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f75a45-fee6-4366-b814-52fa16d24f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:07.221262 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f75a45-fee6-4366-b814-52fa16d24f77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:07.238639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f75a45-fee6-4366-b814-52fa16d24f77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:07.240135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f75a45-fee6-4366-b814-52fa16d24f77 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:07.249695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debecda4-e90e-4371-b5e4-3524a71680ce None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-08f75a45-fee6-4366-b814-52fa16d24f77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:07.258841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:07.260408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:07.262984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:07.262984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:07.262984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:07.262984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:07.262984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 233/465] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:07.273317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f75a45-fee6-4366-b814-52fa16d24f77 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:07.314588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:07.327049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c413c-3730-4afb-91ff-91777f785091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:07.329261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c413c-3730-4afb-91ff-91777f785091 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:07.329515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c413c-3730-4afb-91ff-91777f785091 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:07.329779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c413c-3730-4afb-91ff-91777f785091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:07.348502 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c413c-3730-4afb-91ff-91777f785091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:07.350024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c413c-3730-4afb-91ff-91777f785091 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-560c413c-3730-4afb-91ff-91777f785091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:07.371898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:07.372822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:07.373695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:07.373695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:07.373695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:07.373695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:07.377854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 233/466] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:07.380094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c413c-3730-4afb-91ff-91777f785091 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:07.418907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f75a45-fee6-4366-b814-52fa16d24f77 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:07.448331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:07.469223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79424f74-292a-445e-95b1-ae7be739c2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:07.471250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79424f74-292a-445e-95b1-ae7be739c2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:07.472022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79424f74-292a-445e-95b1-ae7be739c2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:07.474214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79424f74-292a-445e-95b1-ae7be739c2a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:07.475213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79424f74-292a-445e-95b1-ae7be739c2a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:07.475731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79424f74-292a-445e-95b1-ae7be739c2a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:07.478413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79424f74-292a-445e-95b1-ae7be739c2a6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:07.506214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 234/467] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:07.513180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79424f74-292a-445e-95b1-ae7be739c2a6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:07.575864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c413c-3730-4afb-91ff-91777f785091 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:07.611723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae9f81-72ff-4a4b-aa4c-b9bd40903c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:07.612805 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae9f81-72ff-4a4b-aa4c-b9bd40903c26 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:07.613314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae9f81-72ff-4a4b-aa4c-b9bd40903c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:07.613803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae9f81-72ff-4a4b-aa4c-b9bd40903c26 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:07.614708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae9f81-72ff-4a4b-aa4c-b9bd40903c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:07.615213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae9f81-72ff-4a4b-aa4c-b9bd40903c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:07.616806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae9f81-72ff-4a4b-aa4c-b9bd40903c26 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:07.630155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 234/468] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:07.631778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcffa3-f4da-476e-a673-fad68fb70b48 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:07.632260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcffa3-f4da-476e-a673-fad68fb70b48 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:07.632747 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcffa3-f4da-476e-a673-fad68fb70b48 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:07.634471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 235/469] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:07.635366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcffa3-f4da-476e-a673-fad68fb70b48 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:07.646820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:07.656878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79424f74-292a-445e-95b1-ae7be739c2a6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:07.697029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1be183-e14f-4a0f-b5f0-98ce9c08a019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:07.698009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1be183-e14f-4a0f-b5f0-98ce9c08a019 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:07.698424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1be183-e14f-4a0f-b5f0-98ce9c08a019 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:07.698891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1be183-e14f-4a0f-b5f0-98ce9c08a019 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:07.699903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1be183-e14f-4a0f-b5f0-98ce9c08a019 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:07.701657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1be183-e14f-4a0f-b5f0-98ce9c08a019 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:07.726736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 235/470] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:12:07 2022] POST /identity/v3/users => generated 248 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:07.728531 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f9768-42d1-4d6d-a192-84c6e10aa6f3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:07.729072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f9768-42d1-4d6d-a192-84c6e10aa6f3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:07.729503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24f9768-42d1-4d6d-a192-84c6e10aa6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:07.753554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b24f9768-42d1-4d6d-a192-84c6e10aa6f3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:07.755329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b24f9768-42d1-4d6d-a192-84c6e10aa6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:07.756266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:07.765660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcffa3-f4da-476e-a673-fad68fb70b48 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:07.843500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdf39f-2721-4088-9f21-1f492da8e37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:07.844160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdf39f-2721-4088-9f21-1f492da8e37d None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:07.844440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdf39f-2721-4088-9f21-1f492da8e37d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:07.844715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdf39f-2721-4088-9f21-1f492da8e37d None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:07.845409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdf39f-2721-4088-9f21-1f492da8e37d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:07.847211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdf39f-2721-4088-9f21-1f492da8e37d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:07.870052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 236/471] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:07.894172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 236/472] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:12:07 2022] POST /identity/v3/users => generated 248 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:07.906368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f9768-42d1-4d6d-a192-84c6e10aa6f3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:08.012080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:08.026640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a31c1-c81f-414a-9fff-6fdbfbed085c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:08.027174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a31c1-c81f-414a-9fff-6fdbfbed085c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:08.027174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a31c1-c81f-414a-9fff-6fdbfbed085c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:08.027561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24f9768-42d1-4d6d-a192-84c6e10aa6f3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:08.028850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 237/473] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:12:08.035052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3489e27-e286-4b90-aba5-371bf595e112 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:08.035248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3489e27-e286-4b90-aba5-371bf595e112 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:08.035529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3489e27-e286-4b90-aba5-371bf595e112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:08.056759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a3489e27-e286-4b90-aba5-371bf595e112 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:08.058036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3489e27-e286-4b90-aba5-371bf595e112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:08.069076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45b349-e19e-40d5-a080-9ea132727c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:08.069838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45b349-e19e-40d5-a080-9ea132727c78 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:08.070120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45b349-e19e-40d5-a080-9ea132727c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:08.070388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45b349-e19e-40d5-a080-9ea132727c78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:08.082482 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45b349-e19e-40d5-a080-9ea132727c78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:08.084088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45b349-e19e-40d5-a080-9ea132727c78 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bf45b349-e19e-40d5-a080-9ea132727c78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:08.103998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:08.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:08.105804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:08.105804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:08.105804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:08.105804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:08.105804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 237/474] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:08.112125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45b349-e19e-40d5-a080-9ea132727c78 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:08.150890 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 238/475] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:12:08.172721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3489e27-e286-4b90-aba5-371bf595e112 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:08.256789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45b349-e19e-40d5-a080-9ea132727c78 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:08.293955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:08.303643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f417f-9544-4dd9-a309-f71a2aa9325a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:08.304277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f417f-9544-4dd9-a309-f71a2aa9325a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:08.304556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f417f-9544-4dd9-a309-f71a2aa9325a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:08.304734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:08.304881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282f417f-9544-4dd9-a309-f71a2aa9325a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:08.305632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f417f-9544-4dd9-a309-f71a2aa9325a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:08.306116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f417f-9544-4dd9-a309-f71a2aa9325a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:08.307545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282f417f-9544-4dd9-a309-f71a2aa9325a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:08.335248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 238/476] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:08.338510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3489e27-e286-4b90-aba5-371bf595e112 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:08.341436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f417f-9544-4dd9-a309-f71a2aa9325a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:08.416933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a5e42-8721-4c3e-97d8-2531ddd41069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:08.417892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a5e42-8721-4c3e-97d8-2531ddd41069 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:08.418356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a5e42-8721-4c3e-97d8-2531ddd41069 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:08.418766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a5e42-8721-4c3e-97d8-2531ddd41069 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:08.419655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a5e42-8721-4c3e-97d8-2531ddd41069 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:08.421210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545a5e42-8721-4c3e-97d8-2531ddd41069 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:08.436705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.sql.core [None req-545a5e42-8721-4c3e-97d8-2531ddd41069 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 07:12:08.436705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 07:12:08.436705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [parameters: {'id': '801bda6ff4f24bcbbe13e59ff6e864c3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 07:12:08.436705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80169) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-545a5e42-8721-4c3e-97d8-2531ddd41069 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Jul 26 07:12:08.479624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return ref.to_dict() Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 07:12:08.481279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application yield resource Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application next(self.gen) Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise self.value Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application session.commit() Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 07:12:08.483013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.session.flush() Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application compat.raise_( Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 07:12:08.486360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 07:12:08.489793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application conn.query(q) Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application result.read() Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 07:12:08.492970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [parameters: {'id': '801bda6ff4f24bcbbe13e59ff6e864c3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:08.496048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 26 07:12:08.499258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:08.502731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 239/477] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Jul 26 07:12:08 2022] POST /identity/v3/roles => generated 144 bytes in 309 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:08.502731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a5e42-8721-4c3e-97d8-2531ddd41069 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:08.560722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282f417f-9544-4dd9-a309-f71a2aa9325a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:08.602518 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fc498-93c4-404d-983c-e78118411293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:08.603081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fc498-93c4-404d-983c-e78118411293 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:08.603393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fc498-93c4-404d-983c-e78118411293 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:08.603657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177fc498-93c4-404d-983c-e78118411293 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:08.604327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fc498-93c4-404d-983c-e78118411293 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:08.606341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177fc498-93c4-404d-983c-e78118411293 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:08.634517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 239/478] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:12:08 2022] POST /identity/v3/users => generated 249 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:12:08.701018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a5e42-8721-4c3e-97d8-2531ddd41069 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:08.741380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3d061-2385-4cba-9511-823452cdbd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:08.742231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3d061-2385-4cba-9511-823452cdbd3e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:08.742592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3d061-2385-4cba-9511-823452cdbd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:08.742940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3d061-2385-4cba-9511-823452cdbd3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:08.756568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3d061-2385-4cba-9511-823452cdbd3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-48d3d061-2385-4cba-9511-823452cdbd3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:08.762425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:08.765194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:08.767897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 240/479] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:08.778349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fc498-93c4-404d-983c-e78118411293 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:08.830330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:08.888993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177fc498-93c4-404d-983c-e78118411293 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:08.924682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cda231-be42-42fa-8792-a1740daa1387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:08.925514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda231-be42-42fa-8792-a1740daa1387 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:08.925921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda231-be42-42fa-8792-a1740daa1387 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:08.926271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cda231-be42-42fa-8792-a1740daa1387 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:08.927127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cda231-be42-42fa-8792-a1740daa1387 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:08.927524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cda231-be42-42fa-8792-a1740daa1387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:08.928974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cda231-be42-42fa-8792-a1740daa1387 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:08.938315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 240/480] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:09.751761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405e1d9-6b22-49b1-aa7f-59b59a257914 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:09.751761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405e1d9-6b22-49b1-aa7f-59b59a257914 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:09.751761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405e1d9-6b22-49b1-aa7f-59b59a257914 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:09.753031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 241/481] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:09.756971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb02507-001a-4940-bf98-52967f6ee87c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:09.757804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb02507-001a-4940-bf98-52967f6ee87c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:09.758310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb02507-001a-4940-bf98-52967f6ee87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:09.782871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1fb02507-001a-4940-bf98-52967f6ee87c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:09.784652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1fb02507-001a-4940-bf98-52967f6ee87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:09.872189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 241/482] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:09.907610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405e1d9-6b22-49b1-aa7f-59b59a257914 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:09.922265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:10.047521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405e1d9-6b22-49b1-aa7f-59b59a257914 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:10.089829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b7324-a56b-44cf-8ad1-a1d409c585a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:10.090511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b7324-a56b-44cf-8ad1-a1d409c585a9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:10.090792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b7324-a56b-44cf-8ad1-a1d409c585a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:10.091067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b7324-a56b-44cf-8ad1-a1d409c585a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:10.103430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b7324-a56b-44cf-8ad1-a1d409c585a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a31b7324-a56b-44cf-8ad1-a1d409c585a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:10.109015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:10.109902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:10.110705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 242/483] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:10.119750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb02507-001a-4940-bf98-52967f6ee87c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:10.180171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:10.277698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb02507-001a-4940-bf98-52967f6ee87c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:10.320183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb6e1-cef7-4a47-9c66-fb5e599fc96e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:10.320659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb6e1-cef7-4a47-9c66-fb5e599fc96e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:10.321018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180eb6e1-cef7-4a47-9c66-fb5e599fc96e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:10.330682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 243/484] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:10.333780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dbc5f-297d-4456-a2a4-d25a9e2e9d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:10.334635 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dbc5f-297d-4456-a2a4-d25a9e2e9d11 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:10.335089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dbc5f-297d-4456-a2a4-d25a9e2e9d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:10.335534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dbc5f-297d-4456-a2a4-d25a9e2e9d11 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:10.335767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b73a2-1346-4efa-9c5b-704d37d7164e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:10.336129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b73a2-1346-4efa-9c5b-704d37d7164e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:10.336473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97b73a2-1346-4efa-9c5b-704d37d7164e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:10.336986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01dbc5f-297d-4456-a2a4-d25a9e2e9d11 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:10.337456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01dbc5f-297d-4456-a2a4-d25a9e2e9d11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:10.339057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01dbc5f-297d-4456-a2a4-d25a9e2e9d11 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:10.348934 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 242/485] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:10.359333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dbc5f-297d-4456-a2a4-d25a9e2e9d11 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:10.390949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a97b73a2-1346-4efa-9c5b-704d37d7164e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:10.392236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a97b73a2-1346-4efa-9c5b-704d37d7164e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:10.496657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:10.504435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dbc5f-297d-4456-a2a4-d25a9e2e9d11 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:10.570642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff372be4-fc03-4607-93d1-04f683fdb3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:10.571238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff372be4-fc03-4607-93d1-04f683fdb3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:10.571527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff372be4-fc03-4607-93d1-04f683fdb3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:10.571807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff372be4-fc03-4607-93d1-04f683fdb3d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:10.582114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:10.590279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 244/486] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:10.597213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff372be4-fc03-4607-93d1-04f683fdb3d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:10.598735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff372be4-fc03-4607-93d1-04f683fdb3d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:10.614656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b73a2-1346-4efa-9c5b-704d37d7164e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ff372be4-fc03-4607-93d1-04f683fdb3d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:10.626743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:10.627620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:10.628534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:10.628534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:10.628534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:10.628534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:10.630546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 243/487] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:10.637328 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff372be4-fc03-4607-93d1-04f683fdb3d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:10.795476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97b73a2-1346-4efa-9c5b-704d37d7164e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:10.812351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff372be4-fc03-4607-93d1-04f683fdb3d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:10.841008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9a548-a48f-4559-b6a8-e05959b11b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:10.842429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9a548-a48f-4559-b6a8-e05959b11b8c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:10.843102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9a548-a48f-4559-b6a8-e05959b11b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:10.843540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9a548-a48f-4559-b6a8-e05959b11b8c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:10.850300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2072c60-f664-41f6-92f8-91fd6003fb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:10.851061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2072c60-f664-41f6-92f8-91fd6003fb1a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:10.851458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2072c60-f664-41f6-92f8-91fd6003fb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:10.851855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2072c60-f664-41f6-92f8-91fd6003fb1a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:10.852783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2072c60-f664-41f6-92f8-91fd6003fb1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:10.853164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2072c60-f664-41f6-92f8-91fd6003fb1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:10.854770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2072c60-f664-41f6-92f8-91fd6003fb1a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:10.857800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9a548-a48f-4559-b6a8-e05959b11b8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7ee9a548-a48f-4559-b6a8-e05959b11b8c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:10.864544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:10.865463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:10.866289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:10.871344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 245/488] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:10.871344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5855b8-3858-4ed2-b049-b8225c27c387 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:10.871344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5855b8-3858-4ed2-b049-b8225c27c387 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:10.871344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5855b8-3858-4ed2-b049-b8225c27c387 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:10.873814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 246/489] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:10.875577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5855b8-3858-4ed2-b049-b8225c27c387 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:10.878404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 244/490] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:10.880156 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244ffe5-9518-4770-8258-1ec0a13a126f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:10.880597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244ffe5-9518-4770-8258-1ec0a13a126f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:10.880967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244ffe5-9518-4770-8258-1ec0a13a126f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:10.901148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5244ffe5-9518-4770-8258-1ec0a13a126f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:10.902618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5244ffe5-9518-4770-8258-1ec0a13a126f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:10.911935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:10.952783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:10.990108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 245/491] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:10.991174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244ffe5-9518-4770-8258-1ec0a13a126f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:11.003451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5855b8-3858-4ed2-b049-b8225c27c387 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:11.062749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7e2e6-2502-40c2-83cc-ee0537844601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:11.062749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7e2e6-2502-40c2-83cc-ee0537844601 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.062749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7e2e6-2502-40c2-83cc-ee0537844601 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.062749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7e2e6-2502-40c2-83cc-ee0537844601 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.063370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7e2e6-2502-40c2-83cc-ee0537844601 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:11.063370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7e2e6-2502-40c2-83cc-ee0537844601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:11.063370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7e2e6-2502-40c2-83cc-ee0537844601 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:11.079615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 247/492] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:11.086015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7e2e6-2502-40c2-83cc-ee0537844601 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:11.152062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5244ffe5-9518-4770-8258-1ec0a13a126f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:11.188758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3078568-d200-4066-8785-ffd1d0e4be9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:11.189840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3078568-d200-4066-8785-ffd1d0e4be9c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.190478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3078568-d200-4066-8785-ffd1d0e4be9c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.191103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3078568-d200-4066-8785-ffd1d0e4be9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.199879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3078568-d200-4066-8785-ffd1d0e4be9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:11.201932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3078568-d200-4066-8785-ffd1d0e4be9c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-b3078568-d200-4066-8785-ffd1d0e4be9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:11.218080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:11.219047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:11.219976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:11.219976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:11.219976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:11.219976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:11.219976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:11.219976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:11.219976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:11.219976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:11.219976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:11.219976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:11.222588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 246/493] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jul 26 07:12:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:11.223882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3078568-d200-4066-8785-ffd1d0e4be9c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:11.227515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:11.230983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7e2e6-2502-40c2-83cc-ee0537844601 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:11.257178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:11.272527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e3a90-4958-432a-8a76-81d78e4f493f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:11.273194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e3a90-4958-432a-8a76-81d78e4f493f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.273486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e3a90-4958-432a-8a76-81d78e4f493f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.273778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14e3a90-4958-432a-8a76-81d78e4f493f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.287590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14e3a90-4958-432a-8a76-81d78e4f493f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d14e3a90-4958-432a-8a76-81d78e4f493f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:11.293185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:11.294161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:11.294952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:11.299236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 248/494] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:11.299977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e3a90-4958-432a-8a76-81d78e4f493f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:11.347574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3078568-d200-4066-8785-ffd1d0e4be9c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:11.387621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d37ec3f-7200-4c14-a842-b68dca911ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:11.388241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37ec3f-7200-4c14-a842-b68dca911ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.388563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37ec3f-7200-4c14-a842-b68dca911ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.388843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d37ec3f-7200-4c14-a842-b68dca911ce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.400700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37ec3f-7200-4c14-a842-b68dca911ce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:11.402284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d37ec3f-7200-4c14-a842-b68dca911ce3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:11.407889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14e3a90-4958-432a-8a76-81d78e4f493f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6d37ec3f-7200-4c14-a842-b68dca911ce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:11.419574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:11.420459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:11.421294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:11.421294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:11.421294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:11.421294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:11.421294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 247/495] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:11.422365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235678ab-9542-4ba9-baf6-65813536a5b6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.422653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235678ab-9542-4ba9-baf6-65813536a5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.422932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235678ab-9542-4ba9-baf6-65813536a5b6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.426704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 248/496] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:11.426704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235678ab-9542-4ba9-baf6-65813536a5b6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:11.455395 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e993685-daf1-4eb2-8712-0cc1eaa312ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:11.456035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e993685-daf1-4eb2-8712-0cc1eaa312ae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.456388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e993685-daf1-4eb2-8712-0cc1eaa312ae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.456665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e993685-daf1-4eb2-8712-0cc1eaa312ae None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.457475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e993685-daf1-4eb2-8712-0cc1eaa312ae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:11.457817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e993685-daf1-4eb2-8712-0cc1eaa312ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:11.459264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e993685-daf1-4eb2-8712-0cc1eaa312ae None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:11.491939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:11.497599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 249/497] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jul 26 07:12:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:11.498516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e993685-daf1-4eb2-8712-0cc1eaa312ae None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:11.543836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:11.545256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235678ab-9542-4ba9-baf6-65813536a5b6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:11.578677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadb1e1-b0ec-4632-9e92-c6c5e160ec37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:11.579300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadb1e1-b0ec-4632-9e92-c6c5e160ec37 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.579581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadb1e1-b0ec-4632-9e92-c6c5e160ec37 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.579861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadb1e1-b0ec-4632-9e92-c6c5e160ec37 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.580580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadb1e1-b0ec-4632-9e92-c6c5e160ec37 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:11.580881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadb1e1-b0ec-4632-9e92-c6c5e160ec37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:11.582409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadb1e1-b0ec-4632-9e92-c6c5e160ec37 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:11.590932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 249/498] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:11.592268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbf1f6-46b8-4f3a-aeb2-e79758ad85bc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.592555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbf1f6-46b8-4f3a-aeb2-e79758ad85bc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.592835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbbf1f6-46b8-4f3a-aeb2-e79758ad85bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.606031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e993685-daf1-4eb2-8712-0cc1eaa312ae None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:11.611220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ecbbf1f6-46b8-4f3a-aeb2-e79758ad85bc None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:11.612500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ecbbf1f6-46b8-4f3a-aeb2-e79758ad85bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:11.642872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87863349-19e3-4e3f-a2cb-8cc57738e3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:11.643521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87863349-19e3-4e3f-a2cb-8cc57738e3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.643867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87863349-19e3-4e3f-a2cb-8cc57738e3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.644130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87863349-19e3-4e3f-a2cb-8cc57738e3c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.644837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87863349-19e3-4e3f-a2cb-8cc57738e3c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:11.645157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87863349-19e3-4e3f-a2cb-8cc57738e3c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:11.646595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87863349-19e3-4e3f-a2cb-8cc57738e3c8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:11.657925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 250/499] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:11.658667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87863349-19e3-4e3f-a2cb-8cc57738e3c8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:11.683189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 250/500] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:11.684488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2f5ff-5283-4735-b740-7c6c89ca461b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.684813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2f5ff-5283-4735-b740-7c6c89ca461b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.685060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec2f5ff-5283-4735-b740-7c6c89ca461b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.686580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 251/501] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:11.687082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2f5ff-5283-4735-b740-7c6c89ca461b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:11.744800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:11.758150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:11.761345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87863349-19e3-4e3f-a2cb-8cc57738e3c8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:11.796639 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f185f-8bed-4209-8fdb-c8ebfbe29d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:11.797334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f185f-8bed-4209-8fdb-c8ebfbe29d45 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.797547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f185f-8bed-4209-8fdb-c8ebfbe29d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.797852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f185f-8bed-4209-8fdb-c8ebfbe29d45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.801940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec2f5ff-5283-4735-b740-7c6c89ca461b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:11.834964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619365b-0b02-4444-9779-0dc9ac80f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:11.835673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619365b-0b02-4444-9779-0dc9ac80f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.835904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619365b-0b02-4444-9779-0dc9ac80f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.836170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619365b-0b02-4444-9779-0dc9ac80f4c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:11.847763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619365b-0b02-4444-9779-0dc9ac80f4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:11.849331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619365b-0b02-4444-9779-0dc9ac80f4c5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e619365b-0b02-4444-9779-0dc9ac80f4c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:11.865858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:11.866803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:11.867700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:11.867700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:11.867700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:11.867700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:11.867700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 252/502] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:11.868179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619365b-0b02-4444-9779-0dc9ac80f4c5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:11.874682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f185f-8bed-4209-8fdb-c8ebfbe29d45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:11.876210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f185f-8bed-4209-8fdb-c8ebfbe29d45 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:11.966399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:11.969943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619365b-0b02-4444-9779-0dc9ac80f4c5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:11.974867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:11.978581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 251/503] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:11.980241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63953f-76af-461e-9c59-27356bf67d8d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:11.980929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63953f-76af-461e-9c59-27356bf67d8d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:11.981033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff63953f-76af-461e-9c59-27356bf67d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.000471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ff63953f-76af-461e-9c59-27356bf67d8d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:12.001708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ff63953f-76af-461e-9c59-27356bf67d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:12.004756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32533d55-90ec-43e2-9ecb-3ae44dab2b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.005490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32533d55-90ec-43e2-9ecb-3ae44dab2b78 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.005794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32533d55-90ec-43e2-9ecb-3ae44dab2b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.006071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32533d55-90ec-43e2-9ecb-3ae44dab2b78 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.013860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32533d55-90ec-43e2-9ecb-3ae44dab2b78 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:12.015312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32533d55-90ec-43e2-9ecb-3ae44dab2b78 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-32533d55-90ec-43e2-9ecb-3ae44dab2b78 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:12.026745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:12.027699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 253/504] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:12.028825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32533d55-90ec-43e2-9ecb-3ae44dab2b78 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.072437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 252/505] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:12.073283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63953f-76af-461e-9c59-27356bf67d8d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.132606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32533d55-90ec-43e2-9ecb-3ae44dab2b78 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:12.165559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6628cfe-5230-44d3-9e37-2bab6d91c952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.166241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6628cfe-5230-44d3-9e37-2bab6d91c952 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.166523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6628cfe-5230-44d3-9e37-2bab6d91c952 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.166796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6628cfe-5230-44d3-9e37-2bab6d91c952 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.174762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff63953f-76af-461e-9c59-27356bf67d8d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:12.177262 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6628cfe-5230-44d3-9e37-2bab6d91c952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f6628cfe-5230-44d3-9e37-2bab6d91c952 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:12.182720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:12.183764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 254/506] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:12.184905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6628cfe-5230-44d3-9e37-2bab6d91c952 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.204734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:12.207326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:12.214024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69062106-3c13-4bcf-88c3-6c7077ce648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.214641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69062106-3c13-4bcf-88c3-6c7077ce648a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.214929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69062106-3c13-4bcf-88c3-6c7077ce648a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.215198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69062106-3c13-4bcf-88c3-6c7077ce648a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.215909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69062106-3c13-4bcf-88c3-6c7077ce648a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:12.216206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69062106-3c13-4bcf-88c3-6c7077ce648a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:12.217632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69062106-3c13-4bcf-88c3-6c7077ce648a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:12.229483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 253/507] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:12.230403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69062106-3c13-4bcf-88c3-6c7077ce648a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.285845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6628cfe-5230-44d3-9e37-2bab6d91c952 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:12.319743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc7009-b8f1-48ce-922c-c63da551d936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.320390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc7009-b8f1-48ce-922c-c63da551d936 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.320667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc7009-b8f1-48ce-922c-c63da551d936 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.320939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc7009-b8f1-48ce-922c-c63da551d936 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.331341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69062106-3c13-4bcf-88c3-6c7077ce648a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:12.331954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc7009-b8f1-48ce-922c-c63da551d936 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:12.333430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc7009-b8f1-48ce-922c-c63da551d936 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5efc7009-b8f1-48ce-922c-c63da551d936 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:12.348035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:12.349022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:12.350284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:12.350284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:12.350284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:12.350284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:12.350284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 255/508] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:12.350558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc7009-b8f1-48ce-922c-c63da551d936 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.369432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d28bc-0330-464c-9715-59db09c493a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.370126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d28bc-0330-464c-9715-59db09c493a2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.370398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d28bc-0330-464c-9715-59db09c493a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.370659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d28bc-0330-464c-9715-59db09c493a2 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.371448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d28bc-0330-464c-9715-59db09c493a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:12.371750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d28bc-0330-464c-9715-59db09c493a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:12.373197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d28bc-0330-464c-9715-59db09c493a2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:12.390556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 254/509] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:12.391394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d28bc-0330-464c-9715-59db09c493a2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.420084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:12.445979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:12.456082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc7009-b8f1-48ce-922c-c63da551d936 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:12.490031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa507c3c-a0a4-4ab9-bf63-aea8ef299455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.490600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa507c3c-a0a4-4ab9-bf63-aea8ef299455 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.490883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa507c3c-a0a4-4ab9-bf63-aea8ef299455 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.491161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa507c3c-a0a4-4ab9-bf63-aea8ef299455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.495806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d28bc-0330-464c-9715-59db09c493a2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:12.502062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa507c3c-a0a4-4ab9-bf63-aea8ef299455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:12.503638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa507c3c-a0a4-4ab9-bf63-aea8ef299455 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-aa507c3c-a0a4-4ab9-bf63-aea8ef299455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:12.518131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:12.519089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:12.520238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:12.520238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:12.520238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:12.520238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:12.520238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 256/510] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:12.520766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa507c3c-a0a4-4ab9-bf63-aea8ef299455 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.534933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641926be-3dba-427c-8164-ec66e35c20e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.535575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641926be-3dba-427c-8164-ec66e35c20e7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.535851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641926be-3dba-427c-8164-ec66e35c20e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.536120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641926be-3dba-427c-8164-ec66e35c20e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.536848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641926be-3dba-427c-8164-ec66e35c20e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:12.537146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641926be-3dba-427c-8164-ec66e35c20e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:12.538577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641926be-3dba-427c-8164-ec66e35c20e7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:12.546653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 255/511] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:12.547411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641926be-3dba-427c-8164-ec66e35c20e7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.619935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa507c3c-a0a4-4ab9-bf63-aea8ef299455 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:12.642345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:12.647547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641926be-3dba-427c-8164-ec66e35c20e7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:12.654608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e90ad9-17b1-4e3d-bf3a-866f177ce2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.655597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e90ad9-17b1-4e3d-bf3a-866f177ce2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.656091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e90ad9-17b1-4e3d-bf3a-866f177ce2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.656540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e90ad9-17b1-4e3d-bf3a-866f177ce2f0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.660064 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:12.665644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e90ad9-17b1-4e3d-bf3a-866f177ce2f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:12.667139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e90ad9-17b1-4e3d-bf3a-866f177ce2f0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d5e90ad9-17b1-4e3d-bf3a-866f177ce2f0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:12.678394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:12.679352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:12.680274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 257/512] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:12.680850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e90ad9-17b1-4e3d-bf3a-866f177ce2f0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.686355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ab95d-9340-4a78-91c0-794dc04240e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.686998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ab95d-9340-4a78-91c0-794dc04240e1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.687278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ab95d-9340-4a78-91c0-794dc04240e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.687545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ab95d-9340-4a78-91c0-794dc04240e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.688234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ab95d-9340-4a78-91c0-794dc04240e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:12.688536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ab95d-9340-4a78-91c0-794dc04240e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:12.689926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ab95d-9340-4a78-91c0-794dc04240e1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:12.701634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 256/513] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:12.702484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ab95d-9340-4a78-91c0-794dc04240e1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.783011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e90ad9-17b1-4e3d-bf3a-866f177ce2f0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:12.800912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ab95d-9340-4a78-91c0-794dc04240e1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:12.818552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ba48-a565-4f70-8cc6-c881cee9417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.819193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ba48-a565-4f70-8cc6-c881cee9417a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.819463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ba48-a565-4f70-8cc6-c881cee9417a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.819755 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657ba48-a565-4f70-8cc6-c881cee9417a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.836380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715b707-f743-499e-a29b-1d5513820977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:12.837014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715b707-f743-499e-a29b-1d5513820977 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:12.837297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715b707-f743-499e-a29b-1d5513820977 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:12.837577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c715b707-f743-499e-a29b-1d5513820977 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:12.838307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715b707-f743-499e-a29b-1d5513820977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:12.838613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715b707-f743-499e-a29b-1d5513820977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:12.840007 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c715b707-f743-499e-a29b-1d5513820977 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:12.851652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 257/514] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:12.852885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715b707-f743-499e-a29b-1d5513820977 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:12.871240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:12.891326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:12.892489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657ba48-a565-4f70-8cc6-c881cee9417a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:12.894544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657ba48-a565-4f70-8cc6-c881cee9417a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:12.966110 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c715b707-f743-499e-a29b-1d5513820977 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:13.010333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 258/515] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:13.011374 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ba48-a565-4f70-8cc6-c881cee9417a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:13.014466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba6d96-0327-4098-a36f-271cde80332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:13.017434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba6d96-0327-4098-a36f-271cde80332b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:13.017718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba6d96-0327-4098-a36f-271cde80332b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:13.018027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba6d96-0327-4098-a36f-271cde80332b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:13.035019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba6d96-0327-4098-a36f-271cde80332b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:13.037018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba6d96-0327-4098-a36f-271cde80332b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-68ba6d96-0327-4098-a36f-271cde80332b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:13.056442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:13.057370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:13.058241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:13.058241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:13.058241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:13.058241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:13.060397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 258/516] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:13.061384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba6d96-0327-4098-a36f-271cde80332b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:13.119780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:13.142428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657ba48-a565-4f70-8cc6-c881cee9417a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:13.157044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:13.176902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba6d96-0327-4098-a36f-271cde80332b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:13.178333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50600727-6055-4ea7-b373-f79268ee31b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:13.178950 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600727-6055-4ea7-b373-f79268ee31b7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:13.179226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600727-6055-4ea7-b373-f79268ee31b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:13.179502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50600727-6055-4ea7-b373-f79268ee31b7 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:13.180324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50600727-6055-4ea7-b373-f79268ee31b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:13.180624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50600727-6055-4ea7-b373-f79268ee31b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:13.182206 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50600727-6055-4ea7-b373-f79268ee31b7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:13.217501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83069b31-23d1-49ac-a553-c64737222845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:13.218354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83069b31-23d1-49ac-a553-c64737222845 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:13.218740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83069b31-23d1-49ac-a553-c64737222845 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:13.219020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83069b31-23d1-49ac-a553-c64737222845 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:13.225305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 259/517] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jul 26 07:12:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:13.225305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50600727-6055-4ea7-b373-f79268ee31b7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:13.228052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83069b31-23d1-49ac-a553-c64737222845 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:13.229533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83069b31-23d1-49ac-a553-c64737222845 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-83069b31-23d1-49ac-a553-c64737222845 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:13.242311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:13.243186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:13.244011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:13.244011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:13.244011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:13.244011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:13.244011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:13.244011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:13.244011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:13.244011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:13.244011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 259/518] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:13.244463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83069b31-23d1-49ac-a553-c64737222845 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:13.335427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50600727-6055-4ea7-b373-f79268ee31b7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:13.347209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83069b31-23d1-49ac-a553-c64737222845 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:13.369641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea71ff2-9645-4c5a-8529-677ee61c2be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:13.370330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea71ff2-9645-4c5a-8529-677ee61c2be7 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:13.370617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea71ff2-9645-4c5a-8529-677ee61c2be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:13.370889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea71ff2-9645-4c5a-8529-677ee61c2be7 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:13.371584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea71ff2-9645-4c5a-8529-677ee61c2be7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:13.373045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea71ff2-9645-4c5a-8529-677ee61c2be7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:13.381371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28fe1a-f819-46ac-a60a-dece2e28fd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:13.382070 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28fe1a-f819-46ac-a60a-dece2e28fd39 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:13.382371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28fe1a-f819-46ac-a60a-dece2e28fd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:13.382633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da28fe1a-f819-46ac-a60a-dece2e28fd39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:13.384707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:13.395761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28fe1a-f819-46ac-a60a-dece2e28fd39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:13.397254 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da28fe1a-f819-46ac-a60a-dece2e28fd39 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:13.397839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 260/519] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:12:13 2022] POST /identity/v3/users => generated 247 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:13.398769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea71ff2-9645-4c5a-8529-677ee61c2be7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:13.407518 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-da28fe1a-f819-46ac-a60a-dece2e28fd39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:13.415085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:13.416251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:13.417105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:13.417105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:13.417105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:13.417105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:13.417105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 260/520] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:13.417392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28fe1a-f819-46ac-a60a-dece2e28fd39 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:13.584081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea71ff2-9645-4c5a-8529-677ee61c2be7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:13.585951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da28fe1a-f819-46ac-a60a-dece2e28fd39 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:13.632726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f814035-ebf2-4342-83f9-eff785321c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:13.633410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814035-ebf2-4342-83f9-eff785321c94 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:13.633678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814035-ebf2-4342-83f9-eff785321c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:13.633986 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814035-ebf2-4342-83f9-eff785321c94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:13.644245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f209517-107a-482f-b811-1b784762e647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:13.645254 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f209517-107a-482f-b811-1b784762e647 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:13.645753 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f209517-107a-482f-b811-1b784762e647 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:13.646241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f209517-107a-482f-b811-1b784762e647 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:13.647493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f209517-107a-482f-b811-1b784762e647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:13.648121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f209517-107a-482f-b811-1b784762e647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:13.650644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f209517-107a-482f-b811-1b784762e647 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:13.673789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 261/521] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:13.673789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f209517-107a-482f-b811-1b784762e647 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:13.714278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:13.726399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f814035-ebf2-4342-83f9-eff785321c94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:13.728286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f814035-ebf2-4342-83f9-eff785321c94 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:13.742198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:13.796142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f209517-107a-482f-b811-1b784762e647 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:13.838509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48ef7b-5c61-4468-b351-fb5acf4d6363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:13.844081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48ef7b-5c61-4468-b351-fb5acf4d6363 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:13.844257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 261/522] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:13.844680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48ef7b-5c61-4468-b351-fb5acf4d6363 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:13.844680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e48ef7b-5c61-4468-b351-fb5acf4d6363 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:13.844868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f814035-ebf2-4342-83f9-eff785321c94 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:13.844952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48ef7b-5c61-4468-b351-fb5acf4d6363 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:13.844952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48ef7b-5c61-4468-b351-fb5acf4d6363 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:13.844952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e48ef7b-5c61-4468-b351-fb5acf4d6363 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:13.866047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 262/523] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:13.866047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48ef7b-5c61-4468-b351-fb5acf4d6363 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:13.948607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:13.956870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f814035-ebf2-4342-83f9-eff785321c94 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:13.970394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e48ef7b-5c61-4468-b351-fb5acf4d6363 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:13.987833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:13.996809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058839c5-a682-4922-a7e7-d1cd9bc5d250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:13.997699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058839c5-a682-4922-a7e7-d1cd9bc5d250 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:13.998150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058839c5-a682-4922-a7e7-d1cd9bc5d250 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:13.998567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058839c5-a682-4922-a7e7-d1cd9bc5d250 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:13.999423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058839c5-a682-4922-a7e7-d1cd9bc5d250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:13.999735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058839c5-a682-4922-a7e7-d1cd9bc5d250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.001122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058839c5-a682-4922-a7e7-d1cd9bc5d250 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:14.007071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c066c-2a69-4175-a824-aa26a0e2818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:14.007733 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c066c-2a69-4175-a824-aa26a0e2818b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:14.008212 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c066c-2a69-4175-a824-aa26a0e2818b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:14.008647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15c066c-2a69-4175-a824-aa26a0e2818b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:14.015121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 262/524] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:14.016816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c066c-2a69-4175-a824-aa26a0e2818b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.018299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15c066c-2a69-4175-a824-aa26a0e2818b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:14.020474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058839c5-a682-4922-a7e7-d1cd9bc5d250 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c15c066c-2a69-4175-a824-aa26a0e2818b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:14.030608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:14.031595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:14.032467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 263/525] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:14.033087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c066c-2a69-4175-a824-aa26a0e2818b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:14.142625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058839c5-a682-4922-a7e7-d1cd9bc5d250 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:14.151311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15c066c-2a69-4175-a824-aa26a0e2818b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:14.179546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd66179-cf73-4acc-b714-bb513a384216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:14.180197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd66179-cf73-4acc-b714-bb513a384216 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:14.180474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd66179-cf73-4acc-b714-bb513a384216 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:14.180963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd66179-cf73-4acc-b714-bb513a384216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:14.189785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f6bfd-187f-4d85-8c9b-320329e8f188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:14.189785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f6bfd-187f-4d85-8c9b-320329e8f188 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:14.189785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f6bfd-187f-4d85-8c9b-320329e8f188 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:14.190130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f6bfd-187f-4d85-8c9b-320329e8f188 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:14.190902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f6bfd-187f-4d85-8c9b-320329e8f188 None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=6d50427014064f099af87932c77a849a) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:14.191222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f6bfd-187f-4d85-8c9b-320329e8f188 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.192646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f6bfd-187f-4d85-8c9b-320329e8f188 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:14.209796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 264/526] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:14 2022] GET /identity/v3/role_assignments?user.id=6d50427014064f099af87932c77a849a&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:14.209796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f6bfd-187f-4d85-8c9b-320329e8f188 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:14.214649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd66179-cf73-4acc-b714-bb513a384216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.216810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd66179-cf73-4acc-b714-bb513a384216 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-acd66179-cf73-4acc-b714-bb513a384216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:14.247157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:14.248283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:14.249331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:14.249331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:14.249331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:14.249331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:14.253932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 263/527] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:14.254847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd66179-cf73-4acc-b714-bb513a384216 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:14.267537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:14.285078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:14.363610 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f6bfd-187f-4d85-8c9b-320329e8f188 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:14.395799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd66179-cf73-4acc-b714-bb513a384216 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:14.401584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643786a-12ec-4625-9638-d3feb942b1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:14.403236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643786a-12ec-4625-9638-d3feb942b1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:14.403624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643786a-12ec-4625-9638-d3feb942b1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:14.403991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3643786a-12ec-4625-9638-d3feb942b1f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:14.411454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643786a-12ec-4625-9638-d3feb942b1f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.413062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3643786a-12ec-4625-9638-d3feb942b1f0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3643786a-12ec-4625-9638-d3feb942b1f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:14.426480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:14.427582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:14.428626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:14.428626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:14.428626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:14.428626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:14.428626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:14.428626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:14.428626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:14.428626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:14.437828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be66a1-9fcd-4683-afdc-ff605650ba21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:14.437828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be66a1-9fcd-4683-afdc-ff605650ba21 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:14.438098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 265/528] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:14.438207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be66a1-9fcd-4683-afdc-ff605650ba21 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:14.438207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be66a1-9fcd-4683-afdc-ff605650ba21 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:14.438367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643786a-12ec-4625-9638-d3feb942b1f0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:14.438446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be66a1-9fcd-4683-afdc-ff605650ba21 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:14.438446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be66a1-9fcd-4683-afdc-ff605650ba21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.438446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be66a1-9fcd-4683-afdc-ff605650ba21 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:14.457072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 264/529] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Jul 26 07:12:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:14.457946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be66a1-9fcd-4683-afdc-ff605650ba21 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:14.529242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:14.570801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:14.572281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be66a1-9fcd-4683-afdc-ff605650ba21 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:14.575696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3643786a-12ec-4625-9638-d3feb942b1f0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:14.612025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4aed8-7894-48e2-b2d8-222fd4d65216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:14.612810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4aed8-7894-48e2-b2d8-222fd4d65216 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:14.613090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4aed8-7894-48e2-b2d8-222fd4d65216 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:14.613364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd4aed8-7894-48e2-b2d8-222fd4d65216 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:14.614216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4aed8-7894-48e2-b2d8-222fd4d65216 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:14.614527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4aed8-7894-48e2-b2d8-222fd4d65216 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.616141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd4aed8-7894-48e2-b2d8-222fd4d65216 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:14.625508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a957e-f4fe-41a2-88ae-949c64c987d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:14.626540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a957e-f4fe-41a2-88ae-949c64c987d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:14.627016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a957e-f4fe-41a2-88ae-949c64c987d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:14.627478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210a957e-f4fe-41a2-88ae-949c64c987d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:14.628498 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a957e-f4fe-41a2-88ae-949c64c987d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:14.628821 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a957e-f4fe-41a2-88ae-949c64c987d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.630351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210a957e-f4fe-41a2-88ae-949c64c987d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:14.641901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 265/530] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:14.643227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4aed8-7894-48e2-b2d8-222fd4d65216 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:14.648936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 266/531] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:14.649798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a957e-f4fe-41a2-88ae-949c64c987d2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:14.760001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210a957e-f4fe-41a2-88ae-949c64c987d2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:14.777695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd4aed8-7894-48e2-b2d8-222fd4d65216 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:14.795849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13125396-3672-4bf4-9600-06b26af636cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:14.796476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13125396-3672-4bf4-9600-06b26af636cf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:14.796754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13125396-3672-4bf4-9600-06b26af636cf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:14.797017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13125396-3672-4bf4-9600-06b26af636cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:14.797835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13125396-3672-4bf4-9600-06b26af636cf None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=becff3f1dacb43c8a014c95456e86bc6) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:14.798143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13125396-3672-4bf4-9600-06b26af636cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.799626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13125396-3672-4bf4-9600-06b26af636cf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:14.805728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 267/532] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:14 2022] GET /identity/v3/role_assignments?user.id=becff3f1dacb43c8a014c95456e86bc6&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:14.806465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13125396-3672-4bf4-9600-06b26af636cf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:14.808831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:14.827122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab466eb1-9790-4085-8458-05f2213b9667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:14.827782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab466eb1-9790-4085-8458-05f2213b9667 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:14.828511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab466eb1-9790-4085-8458-05f2213b9667 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:14.828790 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab466eb1-9790-4085-8458-05f2213b9667 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:14.839471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:14.926775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13125396-3672-4bf4-9600-06b26af636cf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:14.970648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b55eb-f476-4500-bcc0-545c3852945e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:14.971418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b55eb-f476-4500-bcc0-545c3852945e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:14.971692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b55eb-f476-4500-bcc0-545c3852945e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:14.972052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b55eb-f476-4500-bcc0-545c3852945e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:14.983952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b55eb-f476-4500-bcc0-545c3852945e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.985436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b55eb-f476-4500-bcc0-545c3852945e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:14.986524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab466eb1-9790-4085-8458-05f2213b9667 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:14.988666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab466eb1-9790-4085-8458-05f2213b9667 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ab7b55eb-f476-4500-bcc0-545c3852945e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:15.008085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:15.009014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:15.009882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:15.009882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:15.009882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:15.009882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:15.009882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:15.009882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:15.009882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:15.009882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:15.018292 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 268/533] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:15.023986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b55eb-f476-4500-bcc0-545c3852945e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:15.121419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 266/534] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:15.130803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab466eb1-9790-4085-8458-05f2213b9667 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:15.156474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:15.204031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:15.239320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b55eb-f476-4500-bcc0-545c3852945e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:15.289512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab466eb1-9790-4085-8458-05f2213b9667 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:15.306811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6945652-2481-4791-bdaa-e3836d8bbde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:15.307449 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6945652-2481-4791-bdaa-e3836d8bbde8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:15.307735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6945652-2481-4791-bdaa-e3836d8bbde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:15.308016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6945652-2481-4791-bdaa-e3836d8bbde8 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:15.308790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6945652-2481-4791-bdaa-e3836d8bbde8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:15.309102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6945652-2481-4791-bdaa-e3836d8bbde8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:15.310587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6945652-2481-4791-bdaa-e3836d8bbde8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:15.338736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 269/535] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:15.342820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6945652-2481-4791-bdaa-e3836d8bbde8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:15.362017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64a5a-70bb-4983-9ea7-b9e03e90728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:15.362749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64a5a-70bb-4983-9ea7-b9e03e90728c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:15.363033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64a5a-70bb-4983-9ea7-b9e03e90728c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:15.363303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c64a5a-70bb-4983-9ea7-b9e03e90728c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:15.380006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c64a5a-70bb-4983-9ea7-b9e03e90728c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:15.381497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c64a5a-70bb-4983-9ea7-b9e03e90728c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-02c64a5a-70bb-4983-9ea7-b9e03e90728c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:15.408020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:15.408936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:15.409902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:15.409902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:15.409902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:15.409902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:15.409902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 267/536] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:15.419924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64a5a-70bb-4983-9ea7-b9e03e90728c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:15.535926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6945652-2481-4791-bdaa-e3836d8bbde8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:15.590098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27190ba8-1f87-47c0-a6b0-7e6a390b9a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:15.591146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27190ba8-1f87-47c0-a6b0-7e6a390b9a01 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:15.591667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27190ba8-1f87-47c0-a6b0-7e6a390b9a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:15.592138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27190ba8-1f87-47c0-a6b0-7e6a390b9a01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:15.593148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27190ba8-1f87-47c0-a6b0-7e6a390b9a01 None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=a3c6943686c843e588df14f7c6dffb8a) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:15.593787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27190ba8-1f87-47c0-a6b0-7e6a390b9a01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:15.595496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27190ba8-1f87-47c0-a6b0-7e6a390b9a01 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:15.612105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 270/537] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:15 2022] GET /identity/v3/role_assignments?user.id=a3c6943686c843e588df14f7c6dffb8a&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:15.614737 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:15.646159 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c64a5a-70bb-4983-9ea7-b9e03e90728c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:15.701455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1e40-b7ea-4f03-ab34-ce3f9100e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:15.702145 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1e40-b7ea-4f03-ab34-ce3f9100e5fd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:15.702428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1e40-b7ea-4f03-ab34-ce3f9100e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:15.702703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e1e40-b7ea-4f03-ab34-ce3f9100e5fd None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:15.703419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e1e40-b7ea-4f03-ab34-ce3f9100e5fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:15.703722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e1e40-b7ea-4f03-ab34-ce3f9100e5fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:15.705116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98e1e40-b7ea-4f03-ab34-ce3f9100e5fd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:15.731903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 268/538] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:15.732813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f565a-e42a-4411-a215-6c444160d544 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:15.733173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f565a-e42a-4411-a215-6c444160d544 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:15.736511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f565a-e42a-4411-a215-6c444160d544 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:15.736511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 271/539] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:12:15.737899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1e40-b7ea-4f03-ab34-ce3f9100e5fd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:15.743738 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955b36c-8ee4-4d47-9246-a9cdefb0b2d6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:15.744104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955b36c-8ee4-4d47-9246-a9cdefb0b2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:15.744442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955b36c-8ee4-4d47-9246-a9cdefb0b2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:15.785135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8955b36c-8ee4-4d47-9246-a9cdefb0b2d6 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:15.787286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8955b36c-8ee4-4d47-9246-a9cdefb0b2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:15.792051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:15.892840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e1e40-b7ea-4f03-ab34-ce3f9100e5fd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:15.914060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 272/540] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:12:15.928350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fbc8c-30ba-4aaa-8f8e-49808568c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:15.929100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fbc8c-30ba-4aaa-8f8e-49808568c277 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:15.929442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fbc8c-30ba-4aaa-8f8e-49808568c277 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:15.929798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fbc8c-30ba-4aaa-8f8e-49808568c277 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:15.932702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:15.937474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8fbc8c-30ba-4aaa-8f8e-49808568c277 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:15.939127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8fbc8c-30ba-4aaa-8f8e-49808568c277 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8c8fbc8c-30ba-4aaa-8f8e-49808568c277 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:15.951336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:15.952311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:15.953323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:15.953323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:15.953323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:15.953323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:15.953323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:15.953323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:15.953323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:15.953323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:15.953323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 269/541] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:15.958700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955b36c-8ee4-4d47-9246-a9cdefb0b2d6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:15.958914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fbc8c-30ba-4aaa-8f8e-49808568c277 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:16.153198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8fbc8c-30ba-4aaa-8f8e-49808568c277 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:16.167875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955b36c-8ee4-4d47-9246-a9cdefb0b2d6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:16.234009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c39831-f163-4371-891e-4ece568a9917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:16.235033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c39831-f163-4371-891e-4ece568a9917 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:16.235532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c39831-f163-4371-891e-4ece568a9917 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:16.236025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c39831-f163-4371-891e-4ece568a9917 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:16.246763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c0994-69e4-4920-b6f0-132ab4a58d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:16.247697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c0994-69e4-4920-b6f0-132ab4a58d07 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:16.247981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c0994-69e4-4920-b6f0-132ab4a58d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:16.248254 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519c0994-69e4-4920-b6f0-132ab4a58d07 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:16.249029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c0994-69e4-4920-b6f0-132ab4a58d07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:16.249336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c0994-69e4-4920-b6f0-132ab4a58d07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:16.251130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519c0994-69e4-4920-b6f0-132ab4a58d07 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:16.251373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c39831-f163-4371-891e-4ece568a9917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e8c39831-f163-4371-891e-4ece568a9917 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:16.257612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:16.258681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:16.259516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:16.262813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 270/542] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:16.265575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:16.272044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c39831-f163-4371-891e-4ece568a9917 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:16.284810 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 273/543] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:16.286080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c0994-69e4-4920-b6f0-132ab4a58d07 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:16.372963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:16.429207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c39831-f163-4371-891e-4ece568a9917 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:16.439852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519c0994-69e4-4920-b6f0-132ab4a58d07 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:16.466350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af41626-23e7-4c4c-8942-65a584226c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:16.467010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af41626-23e7-4c4c-8942-65a584226c52 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:16.467288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af41626-23e7-4c4c-8942-65a584226c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:16.467554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af41626-23e7-4c4c-8942-65a584226c52 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:16.468388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af41626-23e7-4c4c-8942-65a584226c52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:16.468688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af41626-23e7-4c4c-8942-65a584226c52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:16.470119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af41626-23e7-4c4c-8942-65a584226c52 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:16.480805 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 271/544] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:16.483028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d21a4-d541-4821-9bc3-a80021b7e978 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:16.483876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d21a4-d541-4821-9bc3-a80021b7e978 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:16.484337 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d21a4-d541-4821-9bc3-a80021b7e978 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:16.486173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 272/545] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:16.486897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d21a4-d541-4821-9bc3-a80021b7e978 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:16.518322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b702035-b8f9-43de-a19b-37b8bc9b53ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:16.520527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b702035-b8f9-43de-a19b-37b8bc9b53ae None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:16.521570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b702035-b8f9-43de-a19b-37b8bc9b53ae None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:16.522279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b702035-b8f9-43de-a19b-37b8bc9b53ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:16.525018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b702035-b8f9-43de-a19b-37b8bc9b53ae None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=1911678a39f749939dc8476617a884fa) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:16.525809 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b702035-b8f9-43de-a19b-37b8bc9b53ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:16.529949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b702035-b8f9-43de-a19b-37b8bc9b53ae None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:16.543158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 274/546] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:16 2022] GET /identity/v3/role_assignments?user.id=1911678a39f749939dc8476617a884fa&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:16.549658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a475d-8f69-4035-925b-9be8304daf6d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:16.549658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a475d-8f69-4035-925b-9be8304daf6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:16.549658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a475d-8f69-4035-925b-9be8304daf6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:16.586226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b96a475d-8f69-4035-925b-9be8304daf6d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:16.588317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b96a475d-8f69-4035-925b-9be8304daf6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:16.669697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d21a4-d541-4821-9bc3-a80021b7e978 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:16.689882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:16.700947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:16.711647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e718c5-3994-44ff-9516-0d117c038251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:16.712302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e718c5-3994-44ff-9516-0d117c038251 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:16.712581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e718c5-3994-44ff-9516-0d117c038251 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:16.712841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e718c5-3994-44ff-9516-0d117c038251 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:16.723867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 275/547] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:16.729787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e718c5-3994-44ff-9516-0d117c038251 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:16.729787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e718c5-3994-44ff-9516-0d117c038251 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f1e718c5-3994-44ff-9516-0d117c038251 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:16.746664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:16.747787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:16.748823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:16.748823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:16.748823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:16.748823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:16.750858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 273/548] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:16.757790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a475d-8f69-4035-925b-9be8304daf6d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:16.759181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e718c5-3994-44ff-9516-0d117c038251 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:16.868386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a475d-8f69-4035-925b-9be8304daf6d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:16.891391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e718c5-3994-44ff-9516-0d117c038251 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:16.904048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23991463-07f7-443a-bbaa-61d39a369a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:16.904686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23991463-07f7-443a-bbaa-61d39a369a25 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:16.904971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23991463-07f7-443a-bbaa-61d39a369a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:16.905244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23991463-07f7-443a-bbaa-61d39a369a25 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:16.905992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23991463-07f7-443a-bbaa-61d39a369a25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:16.906301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23991463-07f7-443a-bbaa-61d39a369a25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:16.907725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23991463-07f7-443a-bbaa-61d39a369a25 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:16.920050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 276/549] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:16.925580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23991463-07f7-443a-bbaa-61d39a369a25 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:16.938731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6b49e-caed-40e4-a879-4c26b2ce331f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:16.939834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6b49e-caed-40e4-a879-4c26b2ce331f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:16.940380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6b49e-caed-40e4-a879-4c26b2ce331f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:16.940945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6b49e-caed-40e4-a879-4c26b2ce331f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:16.961355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6b49e-caed-40e4-a879-4c26b2ce331f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1be6b49e-caed-40e4-a879-4c26b2ce331f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:16.972053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:16.973491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:16.974484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:16.977360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 274/550] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:16.985543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:16.987902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6b49e-caed-40e4-a879-4c26b2ce331f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:17.037139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23991463-07f7-443a-bbaa-61d39a369a25 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:17.040946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:17.086019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5bf74-d091-4080-9a6f-61a523a61b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:17.086695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5bf74-d091-4080-9a6f-61a523a61b9b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:17.086985 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5bf74-d091-4080-9a6f-61a523a61b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:17.087268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf5bf74-d091-4080-9a6f-61a523a61b9b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:17.102613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5bf74-d091-4080-9a6f-61a523a61b9b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:17.104098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf5bf74-d091-4080-9a6f-61a523a61b9b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2bf5bf74-d091-4080-9a6f-61a523a61b9b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:17.129536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:17.130580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:17.131658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 277/551] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:17.139381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5bf74-d091-4080-9a6f-61a523a61b9b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:17.190597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6b49e-caed-40e4-a879-4c26b2ce331f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:17.254770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5403e-a1ab-4b1a-8550-f2cd1d577890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:17.255435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5403e-a1ab-4b1a-8550-f2cd1d577890 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:17.255721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5403e-a1ab-4b1a-8550-f2cd1d577890 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:17.255996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df5403e-a1ab-4b1a-8550-f2cd1d577890 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:17.256737 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5403e-a1ab-4b1a-8550-f2cd1d577890 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:17.257042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5403e-a1ab-4b1a-8550-f2cd1d577890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:17.259455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df5403e-a1ab-4b1a-8550-f2cd1d577890 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:17.281946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 275/552] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:17.288480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5403e-a1ab-4b1a-8550-f2cd1d577890 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:17.321144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf5bf74-d091-4080-9a6f-61a523a61b9b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:17.353521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:17.357727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb4a47-9891-4c40-81ee-27abc6cf4195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:17.358924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb4a47-9891-4c40-81ee-27abc6cf4195 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:17.360186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb4a47-9891-4c40-81ee-27abc6cf4195 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:17.360607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb4a47-9891-4c40-81ee-27abc6cf4195 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:17.361605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb4a47-9891-4c40-81ee-27abc6cf4195 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:17.362163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb4a47-9891-4c40-81ee-27abc6cf4195 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:17.363992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb4a47-9891-4c40-81ee-27abc6cf4195 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:17.383724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 278/553] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jul 26 07:12:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:17.385465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0b19d-684c-4c4f-8031-52d7769721e4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:17.385978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0b19d-684c-4c4f-8031-52d7769721e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:17.386436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d0b19d-684c-4c4f-8031-52d7769721e4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:17.392754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 279/554] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:17.394157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0b19d-684c-4c4f-8031-52d7769721e4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:17.394851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:17.406242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df5403e-a1ab-4b1a-8550-f2cd1d577890 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:17.445466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3879d31-2b5f-4576-8eef-6a7029901e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:17.446492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3879d31-2b5f-4576-8eef-6a7029901e35 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:17.447066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3879d31-2b5f-4576-8eef-6a7029901e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:17.447517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3879d31-2b5f-4576-8eef-6a7029901e35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:17.460489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3879d31-2b5f-4576-8eef-6a7029901e35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:17.462612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3879d31-2b5f-4576-8eef-6a7029901e35 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e3879d31-2b5f-4576-8eef-6a7029901e35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:17.479612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:17.480721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:17.481626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:17.481626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:17.481626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:17.481626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:17.488995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 276/555] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:17.490813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d0472-9c2d-4138-aa62-ad0d55b2417d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:17.491401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d0472-9c2d-4138-aa62-ad0d55b2417d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:17.491873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d0472-9c2d-4138-aa62-ad0d55b2417d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:17.512822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-389d0472-9c2d-4138-aa62-ad0d55b2417d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:17.514426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-389d0472-9c2d-4138-aa62-ad0d55b2417d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:17.520806 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d0b19d-684c-4c4f-8031-52d7769721e4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:17.565960 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d7d3c-26d8-443d-b15c-d7be1db6db32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:17.566599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d7d3c-26d8-443d-b15c-d7be1db6db32 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:17.566886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d7d3c-26d8-443d-b15c-d7be1db6db32 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:17.567165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d7d3c-26d8-443d-b15c-d7be1db6db32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:17.598333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 277/556] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:17.599780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d0472-9c2d-4138-aa62-ad0d55b2417d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:17.618650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:17.670702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d7d3c-26d8-443d-b15c-d7be1db6db32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:17.674362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d7d3c-26d8-443d-b15c-d7be1db6db32 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:17.710282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:17.740993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d0472-9c2d-4138-aa62-ad0d55b2417d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:17.800391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44aefdd-f973-4601-b944-cdf59c64492c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:17.802019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44aefdd-f973-4601-b944-cdf59c64492c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:17.802735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44aefdd-f973-4601-b944-cdf59c64492c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:17.803440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44aefdd-f973-4601-b944-cdf59c64492c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:17.805127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44aefdd-f973-4601-b944-cdf59c64492c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:17.805959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44aefdd-f973-4601-b944-cdf59c64492c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:17.809042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44aefdd-f973-4601-b944-cdf59c64492c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:17.821973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 280/557] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:17.825208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d7d3c-26d8-443d-b15c-d7be1db6db32 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:17.840473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 278/558] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:17.842581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44aefdd-f973-4601-b944-cdf59c64492c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:17.921873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:17.963066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d7d3c-26d8-443d-b15c-d7be1db6db32 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.003980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e175e-03dd-4b59-869d-646e1ce88d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.004671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e175e-03dd-4b59-869d-646e1ce88d6c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.004969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e175e-03dd-4b59-869d-646e1ce88d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.005241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e175e-03dd-4b59-869d-646e1ce88d6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.015193 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44aefdd-f973-4601-b944-cdf59c64492c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.019010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194e175e-03dd-4b59-869d-646e1ce88d6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-194e175e-03dd-4b59-869d-646e1ce88d6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:18.024253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:18.025702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:18.027073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:18.030133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 281/559] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:18.032570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e175e-03dd-4b59-869d-646e1ce88d6c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.057649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51408e2c-b17b-49a6-9483-ebe7a8482849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.058631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51408e2c-b17b-49a6-9483-ebe7a8482849 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.059068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51408e2c-b17b-49a6-9483-ebe7a8482849 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.059530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51408e2c-b17b-49a6-9483-ebe7a8482849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.062288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:18.073695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51408e2c-b17b-49a6-9483-ebe7a8482849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:18.075321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51408e2c-b17b-49a6-9483-ebe7a8482849 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-51408e2c-b17b-49a6-9483-ebe7a8482849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:18.091158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:18.092164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:18.093175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:18.093175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:18.093175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:18.093175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:18.093175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 279/560] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:18.093987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcb72b-2ff6-4947-a9ef-b91a214d3d50 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.093987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcb72b-2ff6-4947-a9ef-b91a214d3d50 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.094148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddcb72b-2ff6-4947-a9ef-b91a214d3d50 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.095617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 280/561] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:18.096168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcb72b-2ff6-4947-a9ef-b91a214d3d50 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.169651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194e175e-03dd-4b59-869d-646e1ce88d6c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.204503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddcb72b-2ff6-4947-a9ef-b91a214d3d50 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.205826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:18.215026 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511bda2-fd3d-405e-b1fd-ce0b1b13da6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.215603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511bda2-fd3d-405e-b1fd-ce0b1b13da6d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.216213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511bda2-fd3d-405e-b1fd-ce0b1b13da6d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.216496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511bda2-fd3d-405e-b1fd-ce0b1b13da6d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.226724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511bda2-fd3d-405e-b1fd-ce0b1b13da6d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:18.228200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e511bda2-fd3d-405e-b1fd-ce0b1b13da6d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e511bda2-fd3d-405e-b1fd-ce0b1b13da6d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:18.246151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:18.247388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 282/562] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:18.248786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511bda2-fd3d-405e-b1fd-ce0b1b13da6d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.252656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454265b-dcb9-4e94-a63e-29a017772b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.253593 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454265b-dcb9-4e94-a63e-29a017772b47 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.253994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454265b-dcb9-4e94-a63e-29a017772b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.254370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454265b-dcb9-4e94-a63e-29a017772b47 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.255202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454265b-dcb9-4e94-a63e-29a017772b47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:18.255938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454265b-dcb9-4e94-a63e-29a017772b47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:18.258405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454265b-dcb9-4e94-a63e-29a017772b47 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:18.269278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 281/563] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:18.270777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07dc0da-6f67-4e12-a63b-9a9bea9a87af None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.271161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07dc0da-6f67-4e12-a63b-9a9bea9a87af None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.271527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07dc0da-6f67-4e12-a63b-9a9bea9a87af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.290452 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f07dc0da-6f67-4e12-a63b-9a9bea9a87af None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:18.291803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f07dc0da-6f67-4e12-a63b-9a9bea9a87af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:18.342836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:18.374908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 282/564] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:18.376335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07dc0da-6f67-4e12-a63b-9a9bea9a87af None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.377480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511bda2-fd3d-405e-b1fd-ce0b1b13da6d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.413243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab2c1b-9fe0-42fb-b041-f5c41c239307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.413920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab2c1b-9fe0-42fb-b041-f5c41c239307 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.414202 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab2c1b-9fe0-42fb-b041-f5c41c239307 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.414476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab2c1b-9fe0-42fb-b041-f5c41c239307 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.415167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab2c1b-9fe0-42fb-b041-f5c41c239307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:18.415469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab2c1b-9fe0-42fb-b041-f5c41c239307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:18.416854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ab2c1b-9fe0-42fb-b041-f5c41c239307 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:18.434746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 283/565] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:18.434746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab2c1b-9fe0-42fb-b041-f5c41c239307 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.490373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:18.500349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07dc0da-6f67-4e12-a63b-9a9bea9a87af None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.537644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623b32e-d715-428f-af62-3f24c04e3d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.538410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623b32e-d715-428f-af62-3f24c04e3d68 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.538753 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623b32e-d715-428f-af62-3f24c04e3d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.539085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623b32e-d715-428f-af62-3f24c04e3d68 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.539880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623b32e-d715-428f-af62-3f24c04e3d68 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:18.540230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623b32e-d715-428f-af62-3f24c04e3d68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:18.541684 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623b32e-d715-428f-af62-3f24c04e3d68 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:18.546849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab2c1b-9fe0-42fb-b041-f5c41c239307 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.560137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 283/566] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:18.560982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623b32e-d715-428f-af62-3f24c04e3d68 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.590629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c273-fd96-4a93-a3d0-2e3ef71b811b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.591294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c273-fd96-4a93-a3d0-2e3ef71b811b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.591576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c273-fd96-4a93-a3d0-2e3ef71b811b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.591849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8c273-fd96-4a93-a3d0-2e3ef71b811b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.596855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:18.603516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8c273-fd96-4a93-a3d0-2e3ef71b811b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:18.604977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8c273-fd96-4a93-a3d0-2e3ef71b811b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6dd8c273-fd96-4a93-a3d0-2e3ef71b811b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:18.620186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:18.621109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:18.622001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:18.622001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:18.622001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:18.622001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:18.622001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 284/567] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:18.623250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c273-fd96-4a93-a3d0-2e3ef71b811b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.674886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623b32e-d715-428f-af62-3f24c04e3d68 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.711557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f26348-a8f1-4360-a70f-71006e724469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.712220 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f26348-a8f1-4360-a70f-71006e724469 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.712486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f26348-a8f1-4360-a70f-71006e724469 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.712752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f26348-a8f1-4360-a70f-71006e724469 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.724391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f26348-a8f1-4360-a70f-71006e724469 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b6f26348-a8f1-4360-a70f-71006e724469 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:18.729870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:18.730888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:18.737483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:18.738728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8c273-fd96-4a93-a3d0-2e3ef71b811b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.738827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 284/568] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:18.738827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f26348-a8f1-4360-a70f-71006e724469 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.740630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:18.777074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8432e2-1b5a-413a-8e64-d3b12f4abd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.777965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8432e2-1b5a-413a-8e64-d3b12f4abd57 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.778096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8432e2-1b5a-413a-8e64-d3b12f4abd57 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.778307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8432e2-1b5a-413a-8e64-d3b12f4abd57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.786220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8432e2-1b5a-413a-8e64-d3b12f4abd57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:18.787597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8432e2-1b5a-413a-8e64-d3b12f4abd57 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5c8432e2-1b5a-413a-8e64-d3b12f4abd57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:18.799363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:18.800308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:18.801193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:18.801193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:18.801193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:18.801193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:18.801193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:18.801193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:18.801193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:18.801193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:18.801193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 285/569] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:18.801673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8432e2-1b5a-413a-8e64-d3b12f4abd57 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.838330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:18.844463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f26348-a8f1-4360-a70f-71006e724469 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.884197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae527e7f-642e-4e74-9015-157b08d18572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.884844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae527e7f-642e-4e74-9015-157b08d18572 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.885118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae527e7f-642e-4e74-9015-157b08d18572 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.885385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae527e7f-642e-4e74-9015-157b08d18572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.895771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae527e7f-642e-4e74-9015-157b08d18572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:18.897237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae527e7f-642e-4e74-9015-157b08d18572 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:18.908101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8432e2-1b5a-413a-8e64-d3b12f4abd57 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ae527e7f-642e-4e74-9015-157b08d18572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:18.912143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:18.913174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:18.914175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:18.914175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:18.914175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:18.914175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:18.914175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 285/570] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:18.914530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae527e7f-642e-4e74-9015-157b08d18572 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.942576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92134b7b-8f4f-4b16-b4c5-1c131c9a3aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:18.943217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92134b7b-8f4f-4b16-b4c5-1c131c9a3aaf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:18.943495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92134b7b-8f4f-4b16-b4c5-1c131c9a3aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:18.943776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92134b7b-8f4f-4b16-b4c5-1c131c9a3aaf None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:18.944469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92134b7b-8f4f-4b16-b4c5-1c131c9a3aaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:18.944773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92134b7b-8f4f-4b16-b4c5-1c131c9a3aaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:18.946207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92134b7b-8f4f-4b16-b4c5-1c131c9a3aaf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:18.958097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 286/571] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:18.959000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92134b7b-8f4f-4b16-b4c5-1c131c9a3aaf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:18.983902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:19.020174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae527e7f-642e-4e74-9015-157b08d18572 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.056836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad31fa-1c15-48e8-8c4c-65730bf93621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.057471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad31fa-1c15-48e8-8c4c-65730bf93621 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.057781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad31fa-1c15-48e8-8c4c-65730bf93621 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.058053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad31fa-1c15-48e8-8c4c-65730bf93621 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.058783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad31fa-1c15-48e8-8c4c-65730bf93621 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:19.059087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad31fa-1c15-48e8-8c4c-65730bf93621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:19.060475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad31fa-1c15-48e8-8c4c-65730bf93621 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:19.062176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92134b7b-8f4f-4b16-b4c5-1c131c9a3aaf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.068602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 286/572] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:19.069895 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85304cb-f2f1-44e2-b40a-5ab9d3a9e3b2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.070191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85304cb-f2f1-44e2-b40a-5ab9d3a9e3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.070464 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85304cb-f2f1-44e2-b40a-5ab9d3a9e3b2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.071694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 287/573] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:19.072375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85304cb-f2f1-44e2-b40a-5ab9d3a9e3b2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:19.086361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:19.100949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656daac-a80e-44ca-acf2-2cb9f1f52863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.101580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656daac-a80e-44ca-acf2-2cb9f1f52863 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.101897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656daac-a80e-44ca-acf2-2cb9f1f52863 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.102165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656daac-a80e-44ca-acf2-2cb9f1f52863 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.102921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656daac-a80e-44ca-acf2-2cb9f1f52863 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:19.103221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656daac-a80e-44ca-acf2-2cb9f1f52863 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:19.104748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656daac-a80e-44ca-acf2-2cb9f1f52863 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:19.116710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 287/574] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:19.117536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656daac-a80e-44ca-acf2-2cb9f1f52863 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:19.184519 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85304cb-f2f1-44e2-b40a-5ab9d3a9e3b2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.201233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:19.218879 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153624e-4c4f-4a00-a587-2adfbb6a23d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.219497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153624e-4c4f-4a00-a587-2adfbb6a23d4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.219772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153624e-4c4f-4a00-a587-2adfbb6a23d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.220044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153624e-4c4f-4a00-a587-2adfbb6a23d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.220730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153624e-4c4f-4a00-a587-2adfbb6a23d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:19.220822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656daac-a80e-44ca-acf2-2cb9f1f52863 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.221027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153624e-4c4f-4a00-a587-2adfbb6a23d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:19.222526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153624e-4c4f-4a00-a587-2adfbb6a23d4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:19.235656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 288/575] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:19.236649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153624e-4c4f-4a00-a587-2adfbb6a23d4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:19.256751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3848d791-88bd-4ea5-ab5d-5ebcc89eaea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.257400 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848d791-88bd-4ea5-ab5d-5ebcc89eaea3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.257675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848d791-88bd-4ea5-ab5d-5ebcc89eaea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.257977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3848d791-88bd-4ea5-ab5d-5ebcc89eaea3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.265684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848d791-88bd-4ea5-ab5d-5ebcc89eaea3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:19.267149 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3848d791-88bd-4ea5-ab5d-5ebcc89eaea3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3848d791-88bd-4ea5-ab5d-5ebcc89eaea3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:19.278664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:19.280167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 288/576] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db4989-ed8e-465c-b87b-9e1756d3c1ec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.281529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db4989-ed8e-465c-b87b-9e1756d3c1ec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.282595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db4989-ed8e-465c-b87b-9e1756d3c1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.299680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-84db4989-ed8e-465c-b87b-9e1756d3c1ec None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:19.301049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-84db4989-ed8e-465c-b87b-9e1756d3c1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:19.312447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:19.344132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153624e-4c4f-4a00-a587-2adfbb6a23d4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.369666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 289/577] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:19.370643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db4989-ed8e-465c-b87b-9e1756d3c1ec None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:19.392300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a9c0-06b2-4832-a34a-ffb02926ef9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.392971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a9c0-06b2-4832-a34a-ffb02926ef9f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.393237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a9c0-06b2-4832-a34a-ffb02926ef9f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.393500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce88a9c0-06b2-4832-a34a-ffb02926ef9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.407943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88a9c0-06b2-4832-a34a-ffb02926ef9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:19.408876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce88a9c0-06b2-4832-a34a-ffb02926ef9f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ce88a9c0-06b2-4832-a34a-ffb02926ef9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:19.424728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:19.425710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:19.426797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:19.426797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:19.426797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:19.426797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:19.427913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 289/578] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:19.428871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a9c0-06b2-4832-a34a-ffb02926ef9f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:19.452778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:19.485453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db4989-ed8e-465c-b87b-9e1756d3c1ec None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.522710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1973c-5393-4822-9d76-6754911da7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.523358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1973c-5393-4822-9d76-6754911da7fe None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.529065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1973c-5393-4822-9d76-6754911da7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.529065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a1973c-5393-4822-9d76-6754911da7fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.529065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1973c-5393-4822-9d76-6754911da7fe None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=64257d34b2df4af1a2016d56be8d44a8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:19.529065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1973c-5393-4822-9d76-6754911da7fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:19.529065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a1973c-5393-4822-9d76-6754911da7fe None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:19.533341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 290/579] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:19 2022] GET /identity/v3/role_assignments?user.id=64257d34b2df4af1a2016d56be8d44a8&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:19.534629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1973c-5393-4822-9d76-6754911da7fe None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:19.536493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce88a9c0-06b2-4832-a34a-ffb02926ef9f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.564430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:19.580386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6df43-55a5-4eb8-a241-997dd23b2b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.581044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6df43-55a5-4eb8-a241-997dd23b2b37 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.581393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6df43-55a5-4eb8-a241-997dd23b2b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.581669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6df43-55a5-4eb8-a241-997dd23b2b37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.589714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6df43-55a5-4eb8-a241-997dd23b2b37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:19.591231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6df43-55a5-4eb8-a241-997dd23b2b37 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-08f6df43-55a5-4eb8-a241-997dd23b2b37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:19.604447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:19.605657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:19.606981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:19.606981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:19.606981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:19.606981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:19.606981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:19.606981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:19.606981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:19.606981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:19.610657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 290/580] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:19.611944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6df43-55a5-4eb8-a241-997dd23b2b37 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:19.687316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a1973c-5393-4822-9d76-6754911da7fe None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.730352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a54a0-4302-4e88-9e88-64b6b0211f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.731007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a54a0-4302-4e88-9e88-64b6b0211f9d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.731340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a54a0-4302-4e88-9e88-64b6b0211f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.731554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a54a0-4302-4e88-9e88-64b6b0211f9d None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.731831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6df43-55a5-4eb8-a241-997dd23b2b37 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.732368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a54a0-4302-4e88-9e88-64b6b0211f9d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:19.732687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a54a0-4302-4e88-9e88-64b6b0211f9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:19.734546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a54a0-4302-4e88-9e88-64b6b0211f9d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:19.737564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:19.755259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 291/581] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jul 26 07:12:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:19.756461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a54a0-4302-4e88-9e88-64b6b0211f9d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:19.779436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c148c2b-9158-439e-8d8e-a81c244f9b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.780084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c148c2b-9158-439e-8d8e-a81c244f9b4b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.780368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c148c2b-9158-439e-8d8e-a81c244f9b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.780644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c148c2b-9158-439e-8d8e-a81c244f9b4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.792644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c148c2b-9158-439e-8d8e-a81c244f9b4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1c148c2b-9158-439e-8d8e-a81c244f9b4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:19.798598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:19.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:19.800749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 291/582] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:19.801434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c148c2b-9158-439e-8d8e-a81c244f9b4b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:19.849469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:19.886032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a54a0-4302-4e88-9e88-64b6b0211f9d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.924132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5238b0-c55c-4250-b893-4e42b9bafaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.925927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5238b0-c55c-4250-b893-4e42b9bafaaf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.925927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5238b0-c55c-4250-b893-4e42b9bafaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.926276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5238b0-c55c-4250-b893-4e42b9bafaaf None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.927364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5238b0-c55c-4250-b893-4e42b9bafaaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:19.927956 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5238b0-c55c-4250-b893-4e42b9bafaaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:19.932559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5238b0-c55c-4250-b893-4e42b9bafaaf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:19.935307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c148c2b-9158-439e-8d8e-a81c244f9b4b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:19.945469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 292/583] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:19.946725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5238b0-c55c-4250-b893-4e42b9bafaaf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:19.983316 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b29e9-248b-49a4-bdbe-ced1058af808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:19.984324 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b29e9-248b-49a4-bdbe-ced1058af808 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:19.984608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b29e9-248b-49a4-bdbe-ced1058af808 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:19.984877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b29e9-248b-49a4-bdbe-ced1058af808 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:19.985685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b29e9-248b-49a4-bdbe-ced1058af808 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:19.986029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b29e9-248b-49a4-bdbe-ced1058af808 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:19.987584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b29e9-248b-49a4-bdbe-ced1058af808 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:20.004867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 292/584] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:20.006070 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b29e9-248b-49a4-bdbe-ced1058af808 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:20.020873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:20.072567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5238b0-c55c-4250-b893-4e42b9bafaaf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:20.122225 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb59c-24b2-4476-a46c-be4fe6ab8162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:20.122894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb59c-24b2-4476-a46c-be4fe6ab8162 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:20.123170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb59c-24b2-4476-a46c-be4fe6ab8162 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:20.123434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28cb59c-24b2-4476-a46c-be4fe6ab8162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:20.127309 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b29e9-248b-49a4-bdbe-ced1058af808 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:20.129228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:20.135517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb59c-24b2-4476-a46c-be4fe6ab8162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:20.136963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28cb59c-24b2-4476-a46c-be4fe6ab8162 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e28cb59c-24b2-4476-a46c-be4fe6ab8162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:20.153646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:20.154825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:20.155740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:20.155740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:20.155740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:20.155740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:20.155740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 293/585] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:20.155740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb59c-24b2-4476-a46c-be4fe6ab8162 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:20.176771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf70630-c3ca-48e4-86bc-107f46bd2c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:20.177584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf70630-c3ca-48e4-86bc-107f46bd2c3f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:20.177994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf70630-c3ca-48e4-86bc-107f46bd2c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:20.178366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf70630-c3ca-48e4-86bc-107f46bd2c3f None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:20.179204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf70630-c3ca-48e4-86bc-107f46bd2c3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:20.179510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf70630-c3ca-48e4-86bc-107f46bd2c3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:20.180927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf70630-c3ca-48e4-86bc-107f46bd2c3f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:20.190549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 293/586] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:20.191667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf70630-c3ca-48e4-86bc-107f46bd2c3f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:20.282255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28cb59c-24b2-4476-a46c-be4fe6ab8162 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:20.290099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:20.302421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf70630-c3ca-48e4-86bc-107f46bd2c3f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:20.321048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3c2e9-ef60-465b-bf3a-a75bd3d4d966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:20.321834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c2e9-ef60-465b-bf3a-a75bd3d4d966 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:20.322197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c2e9-ef60-465b-bf3a-a75bd3d4d966 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:20.322528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3c2e9-ef60-465b-bf3a-a75bd3d4d966 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:20.331583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3c2e9-ef60-465b-bf3a-a75bd3d4d966 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:20.333021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3c2e9-ef60-465b-bf3a-a75bd3d4d966 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:20.338773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112e761-9592-4b4b-8f23-0e2db4224387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:20.339629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112e761-9592-4b4b-8f23-0e2db4224387 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:20.339882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112e761-9592-4b4b-8f23-0e2db4224387 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:20.340195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2112e761-9592-4b4b-8f23-0e2db4224387 None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/6d50427014064f099af87932c77a849a/roles/e998fc875b3b45fc8bcab8bff2f1dc5a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3bb3c2e9-ef60-465b-bf3a-a75bd3d4d966 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:20.345941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:20.346942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:20.347942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:20.347942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:20.347942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:20.347942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:20.347942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:20.347942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:20.347942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:20.347942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:20.347942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 07:12:20.347942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:20.350225 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 294/587] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:20.351089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3c2e9-ef60-465b-bf3a-a75bd3d4d966 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:20.364760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112e761-9592-4b4b-8f23-0e2db4224387 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=6d50427014064f099af87932c77a849a, role_id=e998fc875b3b45fc8bcab8bff2f1dc5a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:20.366331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2112e761-9592-4b4b-8f23-0e2db4224387 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:20.384203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 294/588] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:12:20 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/6d50427014064f099af87932c77a849a/roles/e998fc875b3b45fc8bcab8bff2f1dc5a => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:12:20.395080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112e761-9592-4b4b-8f23-0e2db4224387 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:20.425329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:20.484667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3c2e9-ef60-465b-bf3a-a75bd3d4d966 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:20.526567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f8cce-428f-477f-ac35-1d25bd3ccb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:20.527639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f8cce-428f-477f-ac35-1d25bd3ccb1f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:20.528128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f8cce-428f-477f-ac35-1d25bd3ccb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:20.528690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f8cce-428f-477f-ac35-1d25bd3ccb1f None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:20.529859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f8cce-428f-477f-ac35-1d25bd3ccb1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:20.530394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f8cce-428f-477f-ac35-1d25bd3ccb1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:20.532064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471f8cce-428f-477f-ac35-1d25bd3ccb1f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:20.552444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 295/589] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:20.553989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f8cce-428f-477f-ac35-1d25bd3ccb1f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:20.584904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2112e761-9592-4b4b-8f23-0e2db4224387 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:20.592923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:20.652359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568fcc5-f31d-4075-a0e1-806f1dcdd789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:20.653092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568fcc5-f31d-4075-a0e1-806f1dcdd789 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:20.653498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568fcc5-f31d-4075-a0e1-806f1dcdd789 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:20.653821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568fcc5-f31d-4075-a0e1-806f1dcdd789 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:20.668405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568fcc5-f31d-4075-a0e1-806f1dcdd789 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:20.670131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568fcc5-f31d-4075-a0e1-806f1dcdd789 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:20.686874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f8cce-428f-477f-ac35-1d25bd3ccb1f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b568fcc5-f31d-4075-a0e1-806f1dcdd789 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:20.687243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:20.689053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:20.690059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:20.690059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:20.690059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:20.690059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:20.690059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 295/590] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:20.690059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568fcc5-f31d-4075-a0e1-806f1dcdd789 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:20.723830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad83e6-c852-4a70-be37-e79ba649b114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:20.724475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad83e6-c852-4a70-be37-e79ba649b114 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:20.724756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad83e6-c852-4a70-be37-e79ba649b114 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:20.725033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad83e6-c852-4a70-be37-e79ba649b114 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:20.725882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad83e6-c852-4a70-be37-e79ba649b114 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:20.726192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad83e6-c852-4a70-be37-e79ba649b114 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:20.727647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad83e6-c852-4a70-be37-e79ba649b114 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:20.747804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 296/591] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jul 26 07:12:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:20.748698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad83e6-c852-4a70-be37-e79ba649b114 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:20.834145 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:20.851726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568fcc5-f31d-4075-a0e1-806f1dcdd789 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:20.869437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:20.895233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad83e6-c852-4a70-be37-e79ba649b114 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:20.900519 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1642d-e87e-434b-83b3-f89d550d390e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:20.901439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1642d-e87e-434b-83b3-f89d550d390e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:20.901894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1642d-e87e-434b-83b3-f89d550d390e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:20.902301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1642d-e87e-434b-83b3-f89d550d390e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:20.915321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1642d-e87e-434b-83b3-f89d550d390e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:20.918762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1642d-e87e-434b-83b3-f89d550d390e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b9f1642d-e87e-434b-83b3-f89d550d390e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:20.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:20.938562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:20.940042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:20.940042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:20.940042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:20.940042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:20.940042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:20.940042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:20.940042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:20.940042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:20.941003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 296/592] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:20.941373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1642d-e87e-434b-83b3-f89d550d390e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:20.951546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02427e52-d91c-4072-85aa-a09f9631c436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:20.952205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427e52-d91c-4072-85aa-a09f9631c436 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:20.952489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427e52-d91c-4072-85aa-a09f9631c436 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:20.952762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427e52-d91c-4072-85aa-a09f9631c436 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:20.953492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02427e52-d91c-4072-85aa-a09f9631c436 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:20.953849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02427e52-d91c-4072-85aa-a09f9631c436 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:20.955283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02427e52-d91c-4072-85aa-a09f9631c436 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:20.974112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 297/593] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:20.975009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02427e52-d91c-4072-85aa-a09f9631c436 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:21.098040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1642d-e87e-434b-83b3-f89d550d390e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:21.125728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02427e52-d91c-4072-85aa-a09f9631c436 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:21.136374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:21.151989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a730b-dc3e-4b15-a6e8-86a06482e9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:21.152692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a730b-dc3e-4b15-a6e8-86a06482e9bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:21.153219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a730b-dc3e-4b15-a6e8-86a06482e9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:21.153505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7a730b-dc3e-4b15-a6e8-86a06482e9bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:21.176440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a730b-dc3e-4b15-a6e8-86a06482e9bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:21.178098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7a730b-dc3e-4b15-a6e8-86a06482e9bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:21.178577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca68c2d-e169-4e9e-8098-220a4c80e6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:21.179669 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca68c2d-e169-4e9e-8098-220a4c80e6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:21.180417 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca68c2d-e169-4e9e-8098-220a4c80e6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:21.180903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca68c2d-e169-4e9e-8098-220a4c80e6a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:21.182073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca68c2d-e169-4e9e-8098-220a4c80e6a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:21.182581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca68c2d-e169-4e9e-8098-220a4c80e6a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:21.184331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca68c2d-e169-4e9e-8098-220a4c80e6a8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9b7a730b-dc3e-4b15-a6e8-86a06482e9bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:21.203862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:21.205135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:21.206047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:21.206047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:21.206047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:21.206047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:21.206701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 298/595] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:21.207891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca68c2d-e169-4e9e-8098-220a4c80e6a8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:21.209994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 297/595] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:21.211509 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a730b-dc3e-4b15-a6e8-86a06482e9bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:21.220448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:21.364194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7a730b-dc3e-4b15-a6e8-86a06482e9bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:21.400188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca68c2d-e169-4e9e-8098-220a4c80e6a8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:21.414220 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c7148-0368-46d1-a89f-e158994c1e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:21.415709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c7148-0368-46d1-a89f-e158994c1e77 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:21.416468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c7148-0368-46d1-a89f-e158994c1e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:21.417199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c7148-0368-46d1-a89f-e158994c1e77 None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/becff3f1dacb43c8a014c95456e86bc6/roles/e998fc875b3b45fc8bcab8bff2f1dc5a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:21.444475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c7148-0368-46d1-a89f-e158994c1e77 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=becff3f1dacb43c8a014c95456e86bc6, role_id=e998fc875b3b45fc8bcab8bff2f1dc5a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:21.446184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcf90f-a6f3-4f50-9869-8827d744d871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:21.446920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcf90f-a6f3-4f50-9869-8827d744d871 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:21.447276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcf90f-a6f3-4f50-9869-8827d744d871 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:21.447644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dcf90f-a6f3-4f50-9869-8827d744d871 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:21.448796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c7148-0368-46d1-a89f-e158994c1e77 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:21.457690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcf90f-a6f3-4f50-9869-8827d744d871 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:21.459576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dcf90f-a6f3-4f50-9869-8827d744d871 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:21.470742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 298/596] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:12:21 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/becff3f1dacb43c8a014c95456e86bc6/roles/e998fc875b3b45fc8bcab8bff2f1dc5a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:12:21.471959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c7148-0368-46d1-a89f-e158994c1e77 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e6dcf90f-a6f3-4f50-9869-8827d744d871 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:21.483210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:21.484601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:21.486003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:21.486003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:21.486003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:21.486003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:21.486003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:21.486003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:21.486003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:21.486003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:21.486003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 07:12:21.486003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:21.489358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 299/597] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:21.490609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcf90f-a6f3-4f50-9869-8827d744d871 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:21.510338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:21.592754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c7148-0368-46d1-a89f-e158994c1e77 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:21.600670 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:21.654333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffc047-d2b1-423b-b1ec-79b15060825f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:21.654909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffc047-d2b1-423b-b1ec-79b15060825f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:21.655183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffc047-d2b1-423b-b1ec-79b15060825f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:21.655283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dcf90f-a6f3-4f50-9869-8827d744d871 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:21.655470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffc047-d2b1-423b-b1ec-79b15060825f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:21.656180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffc047-d2b1-423b-b1ec-79b15060825f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:21.656485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffc047-d2b1-423b-b1ec-79b15060825f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:21.659141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffc047-d2b1-423b-b1ec-79b15060825f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:21.681955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 299/598] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:21.683634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e6d45-1fb2-46f1-ab30-a2e7425d1cd0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:21.683961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e6d45-1fb2-46f1-ab30-a2e7425d1cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:21.684200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e6d45-1fb2-46f1-ab30-a2e7425d1cd0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:21.685944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 300/599] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:21.686598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e6d45-1fb2-46f1-ab30-a2e7425d1cd0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:21.705009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154796d1-1482-47ed-a334-e3d790be8a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:21.706208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154796d1-1482-47ed-a334-e3d790be8a58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:21.706208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154796d1-1482-47ed-a334-e3d790be8a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:21.706525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154796d1-1482-47ed-a334-e3d790be8a58 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:21.707406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154796d1-1482-47ed-a334-e3d790be8a58 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:21.707717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154796d1-1482-47ed-a334-e3d790be8a58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:21.709438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154796d1-1482-47ed-a334-e3d790be8a58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:21.748859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 300/600] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Jul 26 07:12:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:21.748859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b3015-4b2b-47ed-b63d-f18e7ab57595 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:21.748859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b3015-4b2b-47ed-b63d-f18e7ab57595 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:21.748859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b3015-4b2b-47ed-b63d-f18e7ab57595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:21.766790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-376b3015-4b2b-47ed-b63d-f18e7ab57595 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:21.768303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-376b3015-4b2b-47ed-b63d-f18e7ab57595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:21.828593 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:21.841369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 301/601] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:21.842616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b3015-4b2b-47ed-b63d-f18e7ab57595 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:21.852023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e6d45-1fb2-46f1-ab30-a2e7425d1cd0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:21.900848 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:21.902921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d148d5-a6c7-40ed-b515-4cf6861ff4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:21.903877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d148d5-a6c7-40ed-b515-4cf6861ff4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:21.905840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d148d5-a6c7-40ed-b515-4cf6861ff4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:21.905840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d148d5-a6c7-40ed-b515-4cf6861ff4c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:21.913918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d148d5-a6c7-40ed-b515-4cf6861ff4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:21.916036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d148d5-a6c7-40ed-b515-4cf6861ff4c9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c8d148d5-a6c7-40ed-b515-4cf6861ff4c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:21.929151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:21.930328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:21.931491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:21.931491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:21.931491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:21.931491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:21.931491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:21.931491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:21.931491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:21.931491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:21.933875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 301/602] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:21.934986 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d148d5-a6c7-40ed-b515-4cf6861ff4c9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:21.952930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b3015-4b2b-47ed-b63d-f18e7ab57595 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:21.989079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e978e-80f8-4a54-9b33-919b91e311cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:21.989768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e978e-80f8-4a54-9b33-919b91e311cf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:21.990048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e978e-80f8-4a54-9b33-919b91e311cf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:21.990327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e978e-80f8-4a54-9b33-919b91e311cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:22.001204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e978e-80f8-4a54-9b33-919b91e311cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:22.002752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e978e-80f8-4a54-9b33-919b91e311cf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9a2e978e-80f8-4a54-9b33-919b91e311cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:22.020870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:22.022064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:22.023173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:22.023173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:22.023173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:22.023173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:22.023173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 302/603] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:22.023173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e978e-80f8-4a54-9b33-919b91e311cf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:22.052215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d148d5-a6c7-40ed-b515-4cf6861ff4c9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:22.090031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18c356-fa7b-469f-902a-93ac4d9994a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:22.090731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18c356-fa7b-469f-902a-93ac4d9994a5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:22.091011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18c356-fa7b-469f-902a-93ac4d9994a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:22.091383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b18c356-fa7b-469f-902a-93ac4d9994a5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:22.103495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b18c356-fa7b-469f-902a-93ac4d9994a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7b18c356-fa7b-469f-902a-93ac4d9994a5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:22.109784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:22.110922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:22.111995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 302/604] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:22.112901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18c356-fa7b-469f-902a-93ac4d9994a5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:22.125992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:22.144635 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e978e-80f8-4a54-9b33-919b91e311cf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:22.148890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:22.200796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c94b1f-f536-4962-9ddb-77932f49986d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:22.201844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c94b1f-f536-4962-9ddb-77932f49986d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:22.202156 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c94b1f-f536-4962-9ddb-77932f49986d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:22.202604 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c94b1f-f536-4962-9ddb-77932f49986d None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:22.203677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c94b1f-f536-4962-9ddb-77932f49986d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:22.204031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c94b1f-f536-4962-9ddb-77932f49986d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:22.205622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c94b1f-f536-4962-9ddb-77932f49986d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:22.225240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 303/605] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:22.226311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c94b1f-f536-4962-9ddb-77932f49986d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:22.233182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b18c356-fa7b-469f-902a-93ac4d9994a5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:22.287182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce169c-c661-4a32-b483-4665488861fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:22.288494 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce169c-c661-4a32-b483-4665488861fe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:22.289179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce169c-c661-4a32-b483-4665488861fe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:22.289893 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce169c-c661-4a32-b483-4665488861fe None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:22.291378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce169c-c661-4a32-b483-4665488861fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:22.292105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce169c-c661-4a32-b483-4665488861fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:22.294613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce169c-c661-4a32-b483-4665488861fe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:22.317024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 303/606] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:22.318831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce169c-c661-4a32-b483-4665488861fe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:22.437428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c94b1f-f536-4962-9ddb-77932f49986d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:22.446261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:22.494828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df07c7-e1aa-4717-b762-8ab9ef269680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:22.496120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df07c7-e1aa-4717-b762-8ab9ef269680 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:22.496606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df07c7-e1aa-4717-b762-8ab9ef269680 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:22.497019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df07c7-e1aa-4717-b762-8ab9ef269680 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:22.497945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df07c7-e1aa-4717-b762-8ab9ef269680 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:22.498395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df07c7-e1aa-4717-b762-8ab9ef269680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:22.499972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3df07c7-e1aa-4717-b762-8ab9ef269680 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:22.511208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 304/607] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:22.512479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df07c7-e1aa-4717-b762-8ab9ef269680 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:22.520459 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce169c-c661-4a32-b483-4665488861fe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:22.560514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fa15a-789e-47ec-98de-b1a67054caed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:22.561729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fa15a-789e-47ec-98de-b1a67054caed None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:22.562205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fa15a-789e-47ec-98de-b1a67054caed None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:22.562623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453fa15a-789e-47ec-98de-b1a67054caed None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/a3c6943686c843e588df14f7c6dffb8a/roles/e998fc875b3b45fc8bcab8bff2f1dc5a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:22.575945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:22.582496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fa15a-789e-47ec-98de-b1a67054caed None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=a3c6943686c843e588df14f7c6dffb8a, role_id=e998fc875b3b45fc8bcab8bff2f1dc5a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:22.584554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453fa15a-789e-47ec-98de-b1a67054caed None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:22.604551 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 304/608] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:12:22 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/a3c6943686c843e588df14f7c6dffb8a/roles/e998fc875b3b45fc8bcab8bff2f1dc5a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:12:22.605969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fa15a-789e-47ec-98de-b1a67054caed None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:22.758223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df07c7-e1aa-4717-b762-8ab9ef269680 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:22.762087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453fa15a-789e-47ec-98de-b1a67054caed None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:22.810269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5c196-2320-4705-8dbb-74e361a5eb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:22.812528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5c196-2320-4705-8dbb-74e361a5eb1a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:22.812850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5c196-2320-4705-8dbb-74e361a5eb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:22.813148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:22.813309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5c196-2320-4705-8dbb-74e361a5eb1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:22.825185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5c196-2320-4705-8dbb-74e361a5eb1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:22.826784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5c196-2320-4705-8dbb-74e361a5eb1a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:22.839084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c83c57-b069-45ff-a12c-f33257d19dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:22.840153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c83c57-b069-45ff-a12c-f33257d19dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:22.840644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c83c57-b069-45ff-a12c-f33257d19dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:22.841257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c83c57-b069-45ff-a12c-f33257d19dc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4ce5c196-2320-4705-8dbb-74e361a5eb1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:22.842500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:22.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:22.844872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:22.844872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:22.844872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:22.844872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:22.844872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:22.844872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:22.844872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:22.844872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:22.844872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 305/609] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:22.845622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68236-70f4-47cf-bc7a-5d1840f93b13 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:22.847385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68236-70f4-47cf-bc7a-5d1840f93b13 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:22.847385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68236-70f4-47cf-bc7a-5d1840f93b13 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:22.848730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 306/610] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:22.853592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb68236-70f4-47cf-bc7a-5d1840f93b13 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:22.857792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c83c57-b069-45ff-a12c-f33257d19dc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:22.859902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c83c57-b069-45ff-a12c-f33257d19dc7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b3c83c57-b069-45ff-a12c-f33257d19dc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:22.880664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:22.881672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:22.882646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:22.882646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:22.882646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:22.882646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:22.882646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 305/611] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:22.883739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b3e21-40c0-45c5-9d4e-11cf04899afb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:22.883987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b3e21-40c0-45c5-9d4e-11cf04899afb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:22.887471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18b3e21-40c0-45c5-9d4e-11cf04899afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:22.910272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e18b3e21-40c0-45c5-9d4e-11cf04899afb None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:22.911901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e18b3e21-40c0-45c5-9d4e-11cf04899afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:22.980052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:23.006447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 306/612] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:23.007330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b3e21-40c0-45c5-9d4e-11cf04899afb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:23.019111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb68236-70f4-47cf-bc7a-5d1840f93b13 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:23.065298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d784205-8d24-4531-b46d-467446ead8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:23.066463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d784205-8d24-4531-b46d-467446ead8ac None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:23.067569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d784205-8d24-4531-b46d-467446ead8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:23.068069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d784205-8d24-4531-b46d-467446ead8ac None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:23.069068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d784205-8d24-4531-b46d-467446ead8ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:23.069910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d784205-8d24-4531-b46d-467446ead8ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:23.072317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d784205-8d24-4531-b46d-467446ead8ac None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:23.089511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 307/613] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:23.091082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d784205-8d24-4531-b46d-467446ead8ac None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:23.136030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:23.142561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18b3e21-40c0-45c5-9d4e-11cf04899afb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:23.200845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8cac2-794d-477c-a050-ec0ef602a9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:23.201865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8cac2-794d-477c-a050-ec0ef602a9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:23.202313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8cac2-794d-477c-a050-ec0ef602a9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:23.202677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8cac2-794d-477c-a050-ec0ef602a9b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:23.203500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8cac2-794d-477c-a050-ec0ef602a9b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:23.203896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8cac2-794d-477c-a050-ec0ef602a9b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:23.205592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f8cac2-794d-477c-a050-ec0ef602a9b2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:23.222133 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 307/614] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:23.223249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8cac2-794d-477c-a050-ec0ef602a9b2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:23.275743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d784205-8d24-4531-b46d-467446ead8ac None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:23.323177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d39ad7-a4dd-4684-a2b5-717f6531b36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:23.324226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d39ad7-a4dd-4684-a2b5-717f6531b36a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:23.324679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d39ad7-a4dd-4684-a2b5-717f6531b36a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:23.325373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d39ad7-a4dd-4684-a2b5-717f6531b36a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:23.340320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d39ad7-a4dd-4684-a2b5-717f6531b36a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:23.346606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8cac2-794d-477c-a050-ec0ef602a9b2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-76d39ad7-a4dd-4684-a2b5-717f6531b36a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:23.347836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:23.348920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 308/615] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:23.363425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d39ad7-a4dd-4684-a2b5-717f6531b36a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:23.377309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:23.407390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d2010-60c4-4640-9c1a-d03963e0e2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:23.408485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2010-60c4-4640-9c1a-d03963e0e2da None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:23.408968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2010-60c4-4640-9c1a-d03963e0e2da None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:23.409475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4d2010-60c4-4640-9c1a-d03963e0e2da None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/1911678a39f749939dc8476617a884fa/roles/e998fc875b3b45fc8bcab8bff2f1dc5a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:23.432458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d2010-60c4-4640-9c1a-d03963e0e2da None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=1911678a39f749939dc8476617a884fa, role_id=e998fc875b3b45fc8bcab8bff2f1dc5a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:23.435575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4d2010-60c4-4640-9c1a-d03963e0e2da None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:23.460974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 308/616] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:12:23 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/1911678a39f749939dc8476617a884fa/roles/e998fc875b3b45fc8bcab8bff2f1dc5a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:12:23.463823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d2010-60c4-4640-9c1a-d03963e0e2da None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:23.485069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:23.496923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d39ad7-a4dd-4684-a2b5-717f6531b36a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:23.539878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bd290-f60a-4a67-8fb6-49ec31ab9f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:23.541059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bd290-f60a-4a67-8fb6-49ec31ab9f9d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:23.541059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bd290-f60a-4a67-8fb6-49ec31ab9f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:23.541407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bd290-f60a-4a67-8fb6-49ec31ab9f9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:23.550969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bd290-f60a-4a67-8fb6-49ec31ab9f9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:23.552550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bd290-f60a-4a67-8fb6-49ec31ab9f9d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-995bd290-f60a-4a67-8fb6-49ec31ab9f9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:23.573244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:23.574719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:23.575898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:23.575898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:23.575898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:23.575898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:23.575898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:23.575898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:23.575898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:23.575898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:23.575898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:23.575898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:23.577711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 309/617] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:23.584376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bd290-f60a-4a67-8fb6-49ec31ab9f9d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:23.613586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4d2010-60c4-4640-9c1a-d03963e0e2da None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:23.658435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f9a60-01a7-492d-a208-4aba5c8b1bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:23.660659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9a60-01a7-492d-a208-4aba5c8b1bdb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:23.661115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9a60-01a7-492d-a208-4aba5c8b1bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:23.661545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f9a60-01a7-492d-a208-4aba5c8b1bdb None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:23.662492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f9a60-01a7-492d-a208-4aba5c8b1bdb None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:23.662963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f9a60-01a7-492d-a208-4aba5c8b1bdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:23.664548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f9a60-01a7-492d-a208-4aba5c8b1bdb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:23.676693 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 309/618] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:23.683111 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f9a60-01a7-492d-a208-4aba5c8b1bdb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:23.727769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:23.792413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:23.818317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bd290-f60a-4a67-8fb6-49ec31ab9f9d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:23.841792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f9a60-01a7-492d-a208-4aba5c8b1bdb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:23.879312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9239e-2746-4536-b70c-6d5dc2adede2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:23.879975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9239e-2746-4536-b70c-6d5dc2adede2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:23.880246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca33320-1632-48af-9ac3-36cf8b8c5f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:23.880433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9239e-2746-4536-b70c-6d5dc2adede2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:23.880513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9239e-2746-4536-b70c-6d5dc2adede2 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:23.880917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33320-1632-48af-9ac3-36cf8b8c5f39 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:23.881199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33320-1632-48af-9ac3-36cf8b8c5f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:23.881369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9239e-2746-4536-b70c-6d5dc2adede2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:23.881471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca33320-1632-48af-9ac3-36cf8b8c5f39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:23.881665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9239e-2746-4536-b70c-6d5dc2adede2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:23.884482 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9239e-2746-4536-b70c-6d5dc2adede2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:23.895058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca33320-1632-48af-9ac3-36cf8b8c5f39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:23.896715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca33320-1632-48af-9ac3-36cf8b8c5f39 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:23.906567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 310/619] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jul 26 07:12:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:23.908535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec36af-687f-41d0-975b-e900c5e49ab7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:23.909052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec36af-687f-41d0-975b-e900c5e49ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:23.909624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec36af-687f-41d0-975b-e900c5e49ab7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:23.911438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 311/620] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:23.912382 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec36af-687f-41d0-975b-e900c5e49ab7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4ca33320-1632-48af-9ac3-36cf8b8c5f39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:23.913058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:23.914566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:23.916277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:23.916277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:23.916277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:23.916277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:23.916277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 310/621] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:23.916804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8058f13-41f4-43a7-87ec-7b15f0cb5679 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:23.916947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8058f13-41f4-43a7-87ec-7b15f0cb5679 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:23.922402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8058f13-41f4-43a7-87ec-7b15f0cb5679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:23.943027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e8058f13-41f4-43a7-87ec-7b15f0cb5679 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:23.944333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e8058f13-41f4-43a7-87ec-7b15f0cb5679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:24.025972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 311/622] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:24.027342 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8058f13-41f4-43a7-87ec-7b15f0cb5679 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:24.056296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec36af-687f-41d0-975b-e900c5e49ab7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:24.092527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:24.106170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:24.110639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a8d91-15ef-479f-b734-97615d293836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:24.111947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a8d91-15ef-479f-b734-97615d293836 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.112225 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a8d91-15ef-479f-b734-97615d293836 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.112494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a8d91-15ef-479f-b734-97615d293836 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.124784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a8d91-15ef-479f-b734-97615d293836 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:24.126977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364a8d91-15ef-479f-b734-97615d293836 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-364a8d91-15ef-479f-b734-97615d293836 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:24.142860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:24.144010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:24.144891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:24.144891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:24.144891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:24.144891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:24.149581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 312/623] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:24.150923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a8d91-15ef-479f-b734-97615d293836 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:24.166060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8058f13-41f4-43a7-87ec-7b15f0cb5679 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:24.203257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035fde8-5a10-40dd-8d27-bd1fa919c76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:24.204277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035fde8-5a10-40dd-8d27-bd1fa919c76a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.204750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035fde8-5a10-40dd-8d27-bd1fa919c76a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.205238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035fde8-5a10-40dd-8d27-bd1fa919c76a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.206228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035fde8-5a10-40dd-8d27-bd1fa919c76a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:24.206936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035fde8-5a10-40dd-8d27-bd1fa919c76a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:24.208545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035fde8-5a10-40dd-8d27-bd1fa919c76a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:24.221661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 312/624] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:24.222917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035fde8-5a10-40dd-8d27-bd1fa919c76a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:24.290755 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a8d91-15ef-479f-b734-97615d293836 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:24.326808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55e1a9-cf18-41b3-8b91-a0a4b47f596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:24.327479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55e1a9-cf18-41b3-8b91-a0a4b47f596b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.327666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55e1a9-cf18-41b3-8b91-a0a4b47f596b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.327943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55e1a9-cf18-41b3-8b91-a0a4b47f596b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.337500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035fde8-5a10-40dd-8d27-bd1fa919c76a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:24.339286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55e1a9-cf18-41b3-8b91-a0a4b47f596b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0e55e1a9-cf18-41b3-8b91-a0a4b47f596b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:24.344779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:24.345769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:24.346853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 313/625] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:24.347664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55e1a9-cf18-41b3-8b91-a0a4b47f596b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:24.357039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:24.381202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c51ef-c00e-41a5-81d9-9cd020ea6904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:24.382121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c51ef-c00e-41a5-81d9-9cd020ea6904 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.382539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c51ef-c00e-41a5-81d9-9cd020ea6904 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.382970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647c51ef-c00e-41a5-81d9-9cd020ea6904 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.383972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c51ef-c00e-41a5-81d9-9cd020ea6904 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:24.384638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:24.385609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c51ef-c00e-41a5-81d9-9cd020ea6904 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:24.387182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647c51ef-c00e-41a5-81d9-9cd020ea6904 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:24.402090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 313/626] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:24.403144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c51ef-c00e-41a5-81d9-9cd020ea6904 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:24.452366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55e1a9-cf18-41b3-8b91-a0a4b47f596b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:24.489663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558756b-a5e9-4017-beb3-5d514aa77ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:24.490384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558756b-a5e9-4017-beb3-5d514aa77ddf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.490959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558756b-a5e9-4017-beb3-5d514aa77ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.490959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6558756b-a5e9-4017-beb3-5d514aa77ddf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.498802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558756b-a5e9-4017-beb3-5d514aa77ddf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:24.500283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6558756b-a5e9-4017-beb3-5d514aa77ddf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6558756b-a5e9-4017-beb3-5d514aa77ddf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:24.513089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:24.514276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:24.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:24.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:24.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:24.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:24.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:24.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:24.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:24.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:24.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:24.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:24.517996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 314/627] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:24.518947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558756b-a5e9-4017-beb3-5d514aa77ddf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:24.520040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647c51ef-c00e-41a5-81d9-9cd020ea6904 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:24.557997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c47572-e3de-43a5-9b90-9c889032d8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:24.558881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47572-e3de-43a5-9b90-9c889032d8aa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.559333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47572-e3de-43a5-9b90-9c889032d8aa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.559755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47572-e3de-43a5-9b90-9c889032d8aa None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.560623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c47572-e3de-43a5-9b90-9c889032d8aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:24.561063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c47572-e3de-43a5-9b90-9c889032d8aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:24.562720 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c47572-e3de-43a5-9b90-9c889032d8aa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:24.573129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 314/628] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:24.574390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c47572-e3de-43a5-9b90-9c889032d8aa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:24.613927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:24.623332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:24.633489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6558756b-a5e9-4017-beb3-5d514aa77ddf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:24.693363 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c47572-e3de-43a5-9b90-9c889032d8aa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:24.694391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcd496-396c-4fe7-8159-f2798baa5ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:24.695398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcd496-396c-4fe7-8159-f2798baa5ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.695893 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcd496-396c-4fe7-8159-f2798baa5ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.696337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdcd496-396c-4fe7-8159-f2798baa5ac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.709941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcd496-396c-4fe7-8159-f2798baa5ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:24.712537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdcd496-396c-4fe7-8159-f2798baa5ac6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:24.733036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2481fb9-7596-43e7-9297-9df780e6ff16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:24.734304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2481fb9-7596-43e7-9297-9df780e6ff16 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.734746 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2481fb9-7596-43e7-9297-9df780e6ff16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.735168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2481fb9-7596-43e7-9297-9df780e6ff16 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.736117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2481fb9-7596-43e7-9297-9df780e6ff16 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2cdcd496-396c-4fe7-8159-f2798baa5ac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:24.736416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:24.737428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:24.738443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:24.738443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:24.738443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:24.738443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:24.738443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:24.738443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:24.738443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:24.738443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:24.739054 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 315/629] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:24.739272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2481fb9-7596-43e7-9297-9df780e6ff16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:24.740563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcd496-396c-4fe7-8159-f2798baa5ac6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:24.741170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2481fb9-7596-43e7-9297-9df780e6ff16 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:24.760707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 315/630] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:24.761865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2481fb9-7596-43e7-9297-9df780e6ff16 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:24.856093 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdcd496-396c-4fe7-8159-f2798baa5ac6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:24.869632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:24.881405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2481fb9-7596-43e7-9297-9df780e6ff16 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:24.893990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f70828-d442-40f3-86f3-9258bb6dfc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:24.894650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f70828-d442-40f3-86f3-9258bb6dfc41 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.894932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f70828-d442-40f3-86f3-9258bb6dfc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.895209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f70828-d442-40f3-86f3-9258bb6dfc41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.906401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f70828-d442-40f3-86f3-9258bb6dfc41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:24.907989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f70828-d442-40f3-86f3-9258bb6dfc41 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-58f70828-d442-40f3-86f3-9258bb6dfc41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:24.922381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:24.923327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:24.924227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:24.924227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:24.924227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:24.924227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:24.924227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 316/631] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:24.924509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f70828-d442-40f3-86f3-9258bb6dfc41 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:24.929703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed47439-5758-4876-8967-e176f462e8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:24.930680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed47439-5758-4876-8967-e176f462e8bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.931227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed47439-5758-4876-8967-e176f462e8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.931511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed47439-5758-4876-8967-e176f462e8bf None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.932282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed47439-5758-4876-8967-e176f462e8bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:24.932593 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed47439-5758-4876-8967-e176f462e8bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:24.934076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed47439-5758-4876-8967-e176f462e8bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:24.939269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:24.953176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 316/632] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:24.954587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33cf9-911b-47a4-a50c-7a31c940103c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:24.954880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33cf9-911b-47a4-a50c-7a31c940103c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:24.955158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e33cf9-911b-47a4-a50c-7a31c940103c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:24.956749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 317/633] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:24.957446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33cf9-911b-47a4-a50c-7a31c940103c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:25.047263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f70828-d442-40f3-86f3-9258bb6dfc41 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:25.066072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e33cf9-911b-47a4-a50c-7a31c940103c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:25.083159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ca00c-b035-4ee9-ac2b-c361105fd570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:25.083791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca00c-b035-4ee9-ac2b-c361105fd570 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:25.084074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca00c-b035-4ee9-ac2b-c361105fd570 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:25.084349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ca00c-b035-4ee9-ac2b-c361105fd570 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:25.100427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3147ac-6a3b-4275-a9b7-a82e83e01d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:25.101064 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3147ac-6a3b-4275-a9b7-a82e83e01d53 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:25.101341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3147ac-6a3b-4275-a9b7-a82e83e01d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:25.101611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3147ac-6a3b-4275-a9b7-a82e83e01d53 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:25.102344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3147ac-6a3b-4275-a9b7-a82e83e01d53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:25.102642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3147ac-6a3b-4275-a9b7-a82e83e01d53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:25.104021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3147ac-6a3b-4275-a9b7-a82e83e01d53 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:25.118268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 318/634] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:25.119400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3147ac-6a3b-4275-a9b7-a82e83e01d53 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:25.135579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:25.166827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ca00c-b035-4ee9-ac2b-c361105fd570 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:25.168295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ca00c-b035-4ee9-ac2b-c361105fd570 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:25.180647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:25.224067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3147ac-6a3b-4275-a9b7-a82e83e01d53 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:25.258383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da732ce-9d9b-429a-ad9b-8bf60c4fe707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:25.259062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da732ce-9d9b-429a-ad9b-8bf60c4fe707 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:25.259348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da732ce-9d9b-429a-ad9b-8bf60c4fe707 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:25.259614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da732ce-9d9b-429a-ad9b-8bf60c4fe707 None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/64257d34b2df4af1a2016d56be8d44a8/roles/e998fc875b3b45fc8bcab8bff2f1dc5a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:25.272905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 317/635] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:25.274221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dbb5f-09fe-4d63-a3a2-28c4bcd455e7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:25.274507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dbb5f-09fe-4d63-a3a2-28c4bcd455e7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:25.274781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765dbb5f-09fe-4d63-a3a2-28c4bcd455e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:25.279214 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da732ce-9d9b-429a-ad9b-8bf60c4fe707 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=64257d34b2df4af1a2016d56be8d44a8, role_id=e998fc875b3b45fc8bcab8bff2f1dc5a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:25.280654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da732ce-9d9b-429a-ad9b-8bf60c4fe707 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:25.293764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-765dbb5f-09fe-4d63-a3a2-28c4bcd455e7 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:25.302101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 319/636] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:12:25 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/64257d34b2df4af1a2016d56be8d44a8/roles/e998fc875b3b45fc8bcab8bff2f1dc5a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:12:25.302352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-765dbb5f-09fe-4d63-a3a2-28c4bcd455e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:25.302452 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da732ce-9d9b-429a-ad9b-8bf60c4fe707 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:25.373054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:25.402010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 318/637] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:25.402887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dbb5f-09fe-4d63-a3a2-28c4bcd455e7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:25.447178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da732ce-9d9b-429a-ad9b-8bf60c4fe707 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:25.467684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:25.494911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1537b3f-ddce-4437-baec-daf1f2a557c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:25.495578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537b3f-ddce-4437-baec-daf1f2a557c5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:25.495859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537b3f-ddce-4437-baec-daf1f2a557c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:25.496131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1537b3f-ddce-4437-baec-daf1f2a557c5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:25.504053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1537b3f-ddce-4437-baec-daf1f2a557c5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:25.505528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1537b3f-ddce-4437-baec-daf1f2a557c5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e1537b3f-ddce-4437-baec-daf1f2a557c5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:25.519293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:25.520735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 320/638] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:25.521670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1537b3f-ddce-4437-baec-daf1f2a557c5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:25.540855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765dbb5f-09fe-4d63-a3a2-28c4bcd455e7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:25.584962 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c13e06-ed1d-407a-a528-5dbdae1ea215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:25.585637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c13e06-ed1d-407a-a528-5dbdae1ea215 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:25.585941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c13e06-ed1d-407a-a528-5dbdae1ea215 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:25.586217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c13e06-ed1d-407a-a528-5dbdae1ea215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:25.611824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c13e06-ed1d-407a-a528-5dbdae1ea215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:25.615546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c13e06-ed1d-407a-a528-5dbdae1ea215 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d5c13e06-ed1d-407a-a528-5dbdae1ea215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:25.637614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:25.638617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:25.639512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:25.639512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:25.639512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:25.639512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:25.639740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1537b3f-ddce-4437-baec-daf1f2a557c5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:25.647624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 319/639] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:25.648955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c13e06-ed1d-407a-a528-5dbdae1ea215 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:25.670881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:25.684417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63c28f-992b-4119-be5e-bb81a4f7e701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:25.685114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63c28f-992b-4119-be5e-bb81a4f7e701 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:25.685395 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63c28f-992b-4119-be5e-bb81a4f7e701 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:25.685671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b63c28f-992b-4119-be5e-bb81a4f7e701 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:25.697786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b63c28f-992b-4119-be5e-bb81a4f7e701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6b63c28f-992b-4119-be5e-bb81a4f7e701 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:25.704388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:25.705427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:25.706296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 321/640] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:25.707137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63c28f-992b-4119-be5e-bb81a4f7e701 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:25.772774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:25.784495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c13e06-ed1d-407a-a528-5dbdae1ea215 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:25.824974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b8167-3277-4b5f-a206-f25dfe82c54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:25.825875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b8167-3277-4b5f-a206-f25dfe82c54b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:25.826256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b8167-3277-4b5f-a206-f25dfe82c54b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:25.826648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9b8167-3277-4b5f-a206-f25dfe82c54b None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:25.827698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b8167-3277-4b5f-a206-f25dfe82c54b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:25.828488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b8167-3277-4b5f-a206-f25dfe82c54b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:25.829802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9b8167-3277-4b5f-a206-f25dfe82c54b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:25.850113 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b63c28f-992b-4119-be5e-bb81a4f7e701 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:25.851112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 320/641] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jul 26 07:12:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:25.852028 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b8167-3277-4b5f-a206-f25dfe82c54b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:25.888591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfa4cc-6a6e-4c3e-8a28-e63ee5860b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:25.889309 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfa4cc-6a6e-4c3e-8a28-e63ee5860b92 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:25.889597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfa4cc-6a6e-4c3e-8a28-e63ee5860b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:25.889905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adfa4cc-6a6e-4c3e-8a28-e63ee5860b92 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:25.890680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfa4cc-6a6e-4c3e-8a28-e63ee5860b92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:25.890990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfa4cc-6a6e-4c3e-8a28-e63ee5860b92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:25.892741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adfa4cc-6a6e-4c3e-8a28-e63ee5860b92 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:25.909774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 322/642] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:25.909774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfa4cc-6a6e-4c3e-8a28-e63ee5860b92 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:25.955160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:25.974331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9b8167-3277-4b5f-a206-f25dfe82c54b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:26.013161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c41a9f-dc09-40a6-8172-eee832b98dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:26.013813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c41a9f-dc09-40a6-8172-eee832b98dcb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.014098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c41a9f-dc09-40a6-8172-eee832b98dcb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.014373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c41a9f-dc09-40a6-8172-eee832b98dcb None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.015103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c41a9f-dc09-40a6-8172-eee832b98dcb None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:26.015409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c41a9f-dc09-40a6-8172-eee832b98dcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:26.016792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c41a9f-dc09-40a6-8172-eee832b98dcb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:26.018409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adfa4cc-6a6e-4c3e-8a28-e63ee5860b92 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:26.025542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 321/643] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:26.026409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c41a9f-dc09-40a6-8172-eee832b98dcb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:26.041338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:26.059983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255896dc-7a08-4c9a-96ac-55f39f31970b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:26.060937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255896dc-7a08-4c9a-96ac-55f39f31970b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.061396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255896dc-7a08-4c9a-96ac-55f39f31970b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.061876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255896dc-7a08-4c9a-96ac-55f39f31970b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.138668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c41a9f-dc09-40a6-8172-eee832b98dcb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:26.141970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255896dc-7a08-4c9a-96ac-55f39f31970b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:26.143642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255896dc-7a08-4c9a-96ac-55f39f31970b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:26.178463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e1608-d8c5-4ed9-80f5-0759a4f8cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:26.179136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e1608-d8c5-4ed9-80f5-0759a4f8cef6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.179420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e1608-d8c5-4ed9-80f5-0759a4f8cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.179691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e1608-d8c5-4ed9-80f5-0759a4f8cef6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.188850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e1608-d8c5-4ed9-80f5-0759a4f8cef6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:26.190506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e1608-d8c5-4ed9-80f5-0759a4f8cef6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:26.194927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-473e1608-d8c5-4ed9-80f5-0759a4f8cef6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:26.203878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:26.204835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:26.206000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:26.206000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:26.206000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:26.206000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:26.206000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:26.206000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:26.206000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:26.206000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:26.206000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 322/644] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:26.207199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2df4a-126a-4d9c-b6b2-c18fbbe11b0b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.207199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2df4a-126a-4d9c-b6b2-c18fbbe11b0b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.207199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2df4a-126a-4d9c-b6b2-c18fbbe11b0b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.212533 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 323/645] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:26.213054 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2df4a-126a-4d9c-b6b2-c18fbbe11b0b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:26.264920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 323/646] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:26.264920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255896dc-7a08-4c9a-96ac-55f39f31970b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:26.291600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:26.330489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2df4a-126a-4d9c-b6b2-c18fbbe11b0b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:26.367833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255896dc-7a08-4c9a-96ac-55f39f31970b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:26.370223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe785b59-ee75-4f2d-b08e-8b1cb86bd380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:26.370908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785b59-ee75-4f2d-b08e-8b1cb86bd380 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.371149 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785b59-ee75-4f2d-b08e-8b1cb86bd380 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.371420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe785b59-ee75-4f2d-b08e-8b1cb86bd380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.382402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe785b59-ee75-4f2d-b08e-8b1cb86bd380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:26.383890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe785b59-ee75-4f2d-b08e-8b1cb86bd380 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fe785b59-ee75-4f2d-b08e-8b1cb86bd380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:26.399045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:26.400186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:26.401124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:26.401124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:26.401124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:26.401124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:26.401124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 324/647] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:26.401958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907ada5-f030-460c-85be-1be682516d9d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.402237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907ada5-f030-460c-85be-1be682516d9d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.402508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e907ada5-f030-460c-85be-1be682516d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.411037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b9cfb-8b02-4fc5-b63b-ad602df163cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:26.411673 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b9cfb-8b02-4fc5-b63b-ad602df163cc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.412093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b9cfb-8b02-4fc5-b63b-ad602df163cc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.412249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b9cfb-8b02-4fc5-b63b-ad602df163cc None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.412973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b9cfb-8b02-4fc5-b63b-ad602df163cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:26.413282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b9cfb-8b02-4fc5-b63b-ad602df163cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:26.414796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b9cfb-8b02-4fc5-b63b-ad602df163cc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:26.422029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e907ada5-f030-460c-85be-1be682516d9d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:26.423327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e907ada5-f030-460c-85be-1be682516d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:26.427781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 324/648] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:26.428789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b9cfb-8b02-4fc5-b63b-ad602df163cc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:26.432726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:26.513437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 325/649] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:26.514592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907ada5-f030-460c-85be-1be682516d9d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:26.560664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b9cfb-8b02-4fc5-b63b-ad602df163cc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:26.577498 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:26.600126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05efe0-2427-4578-b077-f5b33aec5c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:26.600946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05efe0-2427-4578-b077-f5b33aec5c95 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.601336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05efe0-2427-4578-b077-f5b33aec5c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.601707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05efe0-2427-4578-b077-f5b33aec5c95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.612949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05efe0-2427-4578-b077-f5b33aec5c95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:26.614583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc05efe0-2427-4578-b077-f5b33aec5c95 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:26.626298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e907ada5-f030-460c-85be-1be682516d9d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-dc05efe0-2427-4578-b077-f5b33aec5c95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:26.629384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:26.630488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:26.631505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:26.631505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:26.631505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:26.631505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:26.634742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 325/650] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:26.634742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05efe0-2427-4578-b077-f5b33aec5c95 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:26.666086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a4ea2-3275-468e-9e69-756f28b50a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:26.666714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a4ea2-3275-468e-9e69-756f28b50a42 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.666990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a4ea2-3275-468e-9e69-756f28b50a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.667261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a4ea2-3275-468e-9e69-756f28b50a42 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.667998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a4ea2-3275-468e-9e69-756f28b50a42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:26.668298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a4ea2-3275-468e-9e69-756f28b50a42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:26.669691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a4ea2-3275-468e-9e69-756f28b50a42 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:26.682944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 326/651] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:26.683644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a4ea2-3275-468e-9e69-756f28b50a42 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:26.708750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:26.756594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc05efe0-2427-4578-b077-f5b33aec5c95 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:26.789943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f1119-8da8-4bbb-8e70-05ddd3ab6d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:26.790488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f1119-8da8-4bbb-8e70-05ddd3ab6d4c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.790764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f1119-8da8-4bbb-8e70-05ddd3ab6d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.791034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f1119-8da8-4bbb-8e70-05ddd3ab6d4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.791821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f1119-8da8-4bbb-8e70-05ddd3ab6d4c None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=6d50427014064f099af87932c77a849a) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:26.792127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f1119-8da8-4bbb-8e70-05ddd3ab6d4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:26.793583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f1119-8da8-4bbb-8e70-05ddd3ab6d4c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:26.795259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a4ea2-3275-468e-9e69-756f28b50a42 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:26.800142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 326/652] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:26 2022] GET /identity/v3/role_assignments?user.id=6d50427014064f099af87932c77a849a&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:26.801004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f1119-8da8-4bbb-8e70-05ddd3ab6d4c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:26.820920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:26.864197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cc897-4fe8-4437-b5ec-7b1d2d63aade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:26.866908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cc897-4fe8-4437-b5ec-7b1d2d63aade None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.867710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cc897-4fe8-4437-b5ec-7b1d2d63aade None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.868547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312cc897-4fe8-4437-b5ec-7b1d2d63aade None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.887790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312cc897-4fe8-4437-b5ec-7b1d2d63aade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-312cc897-4fe8-4437-b5ec-7b1d2d63aade None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:26.895223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:26.896443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:26.897529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:26.904185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 327/653] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:26.905304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cc897-4fe8-4437-b5ec-7b1d2d63aade None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:26.946345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f1119-8da8-4bbb-8e70-05ddd3ab6d4c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:26.977041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:26.996845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d81c9-6678-41de-bcd4-4fe7fb5d8f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:26.997485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d81c9-6678-41de-bcd4-4fe7fb5d8f10 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:26.998242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d81c9-6678-41de-bcd4-4fe7fb5d8f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:26.998242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d81c9-6678-41de-bcd4-4fe7fb5d8f10 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:26.998758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d81c9-6678-41de-bcd4-4fe7fb5d8f10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:26.999060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d81c9-6678-41de-bcd4-4fe7fb5d8f10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:27.000450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d81c9-6678-41de-bcd4-4fe7fb5d8f10 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:27.014477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 327/654] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:27.015437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d81c9-6678-41de-bcd4-4fe7fb5d8f10 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:27.049207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312cc897-4fe8-4437-b5ec-7b1d2d63aade None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:27.097922 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b68f66-133d-4ee0-9967-9d1d7e77ba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:27.098629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b68f66-133d-4ee0-9967-9d1d7e77ba9b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.098872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b68f66-133d-4ee0-9967-9d1d7e77ba9b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.099149 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b68f66-133d-4ee0-9967-9d1d7e77ba9b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.116923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b68f66-133d-4ee0-9967-9d1d7e77ba9b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:27.119911 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b68f66-133d-4ee0-9967-9d1d7e77ba9b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:27.131834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d81c9-6678-41de-bcd4-4fe7fb5d8f10 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-35b68f66-133d-4ee0-9967-9d1d7e77ba9b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:27.145305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:27.146345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:27.147275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:27.147275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:27.147275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:27.147275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:27.147275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:27.147275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:27.147275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:27.147275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:27.147275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 26 07:12:27.147275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:27.148309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 328/655] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:27.150324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b68f66-133d-4ee0-9967-9d1d7e77ba9b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:27.173915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff2a42-f107-4100-8700-496b727b2783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:27.174563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2a42-f107-4100-8700-496b727b2783 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.174846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2a42-f107-4100-8700-496b727b2783 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.175195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2a42-f107-4100-8700-496b727b2783 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.175928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff2a42-f107-4100-8700-496b727b2783 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:27.176248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff2a42-f107-4100-8700-496b727b2783 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:27.177621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff2a42-f107-4100-8700-496b727b2783 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:27.188179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 328/656] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:27.189051 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff2a42-f107-4100-8700-496b727b2783 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:27.196389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:27.236734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:27.276459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b68f66-133d-4ee0-9967-9d1d7e77ba9b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:27.297101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff2a42-f107-4100-8700-496b727b2783 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:27.314567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da76ade-88b7-4661-b3a2-d5aae3ef0551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:27.315666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da76ade-88b7-4661-b3a2-d5aae3ef0551 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.316205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da76ade-88b7-4661-b3a2-d5aae3ef0551 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.316680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da76ade-88b7-4661-b3a2-d5aae3ef0551 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.324941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da76ade-88b7-4661-b3a2-d5aae3ef0551 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:27.326822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da76ade-88b7-4661-b3a2-d5aae3ef0551 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:27.332893 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0df7be-5642-465e-b370-58c7e8aa7f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:27.333536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0df7be-5642-465e-b370-58c7e8aa7f81 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.333864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0df7be-5642-465e-b370-58c7e8aa7f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.334142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0df7be-5642-465e-b370-58c7e8aa7f81 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.334955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0df7be-5642-465e-b370-58c7e8aa7f81 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:27.335266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0df7be-5642-465e-b370-58c7e8aa7f81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:27.336713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0df7be-5642-465e-b370-58c7e8aa7f81 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7da76ade-88b7-4661-b3a2-d5aae3ef0551 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:27.341826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:27.342751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:27.343639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:27.343639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:27.343639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:27.343639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:27.343639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:27.343639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:27.343639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:27.343639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:27.346310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 329/657] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:27.347382 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da76ade-88b7-4661-b3a2-d5aae3ef0551 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:27.366021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 329/658] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Jul 26 07:12:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:27.367008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0df7be-5642-465e-b370-58c7e8aa7f81 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:27.456305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:27.466537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da76ade-88b7-4661-b3a2-d5aae3ef0551 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:27.478794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:27.482481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0df7be-5642-465e-b370-58c7e8aa7f81 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:27.503032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9691d-c3ff-477e-ac35-83dafcb08375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:27.503755 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9691d-c3ff-477e-ac35-83dafcb08375 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.504032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9691d-c3ff-477e-ac35-83dafcb08375 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.504303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9691d-c3ff-477e-ac35-83dafcb08375 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.515866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9691d-c3ff-477e-ac35-83dafcb08375 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:27.517299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9691d-c3ff-477e-ac35-83dafcb08375 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:27.520707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf812c4-d19b-46e6-b049-28c063d82926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:27.521356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf812c4-d19b-46e6-b049-28c063d82926 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.521644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf812c4-d19b-46e6-b049-28c063d82926 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.521962 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf812c4-d19b-46e6-b049-28c063d82926 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.522726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf812c4-d19b-46e6-b049-28c063d82926 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:27.523037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf812c4-d19b-46e6-b049-28c063d82926 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:27.524967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf812c4-d19b-46e6-b049-28c063d82926 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d5a9691d-c3ff-477e-ac35-83dafcb08375 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:27.531818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:27.532720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:27.533600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:27.533600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:27.533600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:27.533600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:27.542705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 330/659] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:27.542870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 330/660] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:27.542966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9691d-c3ff-477e-ac35-83dafcb08375 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:27.543042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab393636-2a6d-403e-a30c-a0d77b3830cc None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.543042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab393636-2a6d-403e-a30c-a0d77b3830cc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.543042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab393636-2a6d-403e-a30c-a0d77b3830cc None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.543042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 331/661] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:27.543042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab393636-2a6d-403e-a30c-a0d77b3830cc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:27.654626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab393636-2a6d-403e-a30c-a0d77b3830cc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:27.655538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9691d-c3ff-477e-ac35-83dafcb08375 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:27.693975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a68819-2bd6-4e67-b944-a02b9fa067a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:27.694717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a68819-2bd6-4e67-b944-a02b9fa067a3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.694981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398b258-0d63-4860-b5cc-c445daa7a2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:27.695106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a68819-2bd6-4e67-b944-a02b9fa067a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.695420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a68819-2bd6-4e67-b944-a02b9fa067a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.695599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398b258-0d63-4860-b5cc-c445daa7a2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.695875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398b258-0d63-4860-b5cc-c445daa7a2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.696150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398b258-0d63-4860-b5cc-c445daa7a2e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.696839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398b258-0d63-4860-b5cc-c445daa7a2e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:27.697137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398b258-0d63-4860-b5cc-c445daa7a2e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:27.698610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398b258-0d63-4860-b5cc-c445daa7a2e8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:27.715195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 332/662] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:27.716061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398b258-0d63-4860-b5cc-c445daa7a2e8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:27.717155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:27.750458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:27.777582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a68819-2bd6-4e67-b944-a02b9fa067a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:27.779099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a68819-2bd6-4e67-b944-a02b9fa067a3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:27.835649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398b258-0d63-4860-b5cc-c445daa7a2e8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:27.875180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3d4be-9955-47a1-8754-752f93aae445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:27.875813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3d4be-9955-47a1-8754-752f93aae445 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.876189 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3d4be-9955-47a1-8754-752f93aae445 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.876411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3d4be-9955-47a1-8754-752f93aae445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.877162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3d4be-9955-47a1-8754-752f93aae445 None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=becff3f1dacb43c8a014c95456e86bc6) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:27.877476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3d4be-9955-47a1-8754-752f93aae445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:27.878976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3d4be-9955-47a1-8754-752f93aae445 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:27.884952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 333/663] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:27 2022] GET /identity/v3/role_assignments?user.id=becff3f1dacb43c8a014c95456e86bc6&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:27.886235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ad42a-52d6-422b-9e48-f4e344d93a84 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:27.886531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ad42a-52d6-422b-9e48-f4e344d93a84 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:27.886805 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ad42a-52d6-422b-9e48-f4e344d93a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:27.897001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 331/664] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:27.898247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a68819-2bd6-4e67-b944-a02b9fa067a3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:27.911701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5b1ad42a-52d6-422b-9e48-f4e344d93a84 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:27.912878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5b1ad42a-52d6-422b-9e48-f4e344d93a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:27.955647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:28.013787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 334/665] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:28.013787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ad42a-52d6-422b-9e48-f4e344d93a84 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:28.030627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:28.049470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a68819-2bd6-4e67-b944-a02b9fa067a3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:28.111929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135b56b-d67e-4abf-9405-b564cccd34c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:28.113355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135b56b-d67e-4abf-9405-b564cccd34c0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:28.113918 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135b56b-d67e-4abf-9405-b564cccd34c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:28.114352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135b56b-d67e-4abf-9405-b564cccd34c0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:28.132456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135b56b-d67e-4abf-9405-b564cccd34c0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:28.135107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135b56b-d67e-4abf-9405-b564cccd34c0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2135b56b-d67e-4abf-9405-b564cccd34c0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:28.155474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:28.156505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:28.157658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:28.157658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:28.157658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:28.157658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:28.157658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:28.157658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:28.157658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:28.157658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:28.157658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 26 07:12:28.157658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:28.158948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 332/666] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:28.160338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135b56b-d67e-4abf-9405-b564cccd34c0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:28.163500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ad42a-52d6-422b-9e48-f4e344d93a84 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:28.217428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbdb98-73b8-4ea5-b71e-ce3cb14a0b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:28.218159 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbdb98-73b8-4ea5-b71e-ce3cb14a0b11 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:28.218447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbdb98-73b8-4ea5-b71e-ce3cb14a0b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:28.218721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbdb98-73b8-4ea5-b71e-ce3cb14a0b11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:28.230661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbdb98-73b8-4ea5-b71e-ce3cb14a0b11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:28.232381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9dbdb98-73b8-4ea5-b71e-ce3cb14a0b11 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d9dbdb98-73b8-4ea5-b71e-ce3cb14a0b11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:28.256841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:28.258361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:28.259821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:28.259821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:28.259821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:28.259821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:28.265963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 335/667] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:28.267626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbdb98-73b8-4ea5-b71e-ce3cb14a0b11 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:28.298520 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135b56b-d67e-4abf-9405-b564cccd34c0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:28.302065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:28.349341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62d1a3-bb89-4e45-a6b4-a1760bb3af16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:28.350452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62d1a3-bb89-4e45-a6b4-a1760bb3af16 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:28.350999 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62d1a3-bb89-4e45-a6b4-a1760bb3af16 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:28.351544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62d1a3-bb89-4e45-a6b4-a1760bb3af16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:28.369611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62d1a3-bb89-4e45-a6b4-a1760bb3af16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:28.370989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4e62d1a3-bb89-4e45-a6b4-a1760bb3af16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:28.379141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:28.380271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:28.381304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 333/668] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:28.386864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62d1a3-bb89-4e45-a6b4-a1760bb3af16 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:28.417800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbdb98-73b8-4ea5-b71e-ce3cb14a0b11 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:28.465836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8fe0e-86da-4f8d-be63-9b92179a2f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:28.466878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8fe0e-86da-4f8d-be63-9b92179a2f37 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:28.467355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8fe0e-86da-4f8d-be63-9b92179a2f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:28.467834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c8fe0e-86da-4f8d-be63-9b92179a2f37 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:28.468868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8fe0e-86da-4f8d-be63-9b92179a2f37 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:28.469378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8fe0e-86da-4f8d-be63-9b92179a2f37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:28.471486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c8fe0e-86da-4f8d-be63-9b92179a2f37 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:28.498275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 336/669] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Jul 26 07:12:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:28.499621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8fe0e-86da-4f8d-be63-9b92179a2f37 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:28.501112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62d1a3-bb89-4e45-a6b4-a1760bb3af16 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:28.537677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f031140-0fe4-4c06-a37b-ddc2fe254c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:28.538346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f031140-0fe4-4c06-a37b-ddc2fe254c1b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:28.538680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f031140-0fe4-4c06-a37b-ddc2fe254c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:28.538964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f031140-0fe4-4c06-a37b-ddc2fe254c1b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:28.540526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f031140-0fe4-4c06-a37b-ddc2fe254c1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:28.540526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f031140-0fe4-4c06-a37b-ddc2fe254c1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:28.541527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f031140-0fe4-4c06-a37b-ddc2fe254c1b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:28.560509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 334/670] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:28.560509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f031140-0fe4-4c06-a37b-ddc2fe254c1b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:28.586244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:28.631110 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c8fe0e-86da-4f8d-be63-9b92179a2f37 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:28.659180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:28.665149 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f031140-0fe4-4c06-a37b-ddc2fe254c1b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:28.673999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41bddf-0749-4d9b-990c-13144af83225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:28.674983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41bddf-0749-4d9b-990c-13144af83225 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:28.675491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41bddf-0749-4d9b-990c-13144af83225 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:28.675959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41bddf-0749-4d9b-990c-13144af83225 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:28.676981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41bddf-0749-4d9b-990c-13144af83225 None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:28.677461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41bddf-0749-4d9b-990c-13144af83225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:28.679210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f41bddf-0749-4d9b-990c-13144af83225 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:28.689596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 337/671] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:28.690996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41bddf-0749-4d9b-990c-13144af83225 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:28.703013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794e5fd-d14c-47ff-8cb7-8609229d6caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:28.703645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794e5fd-d14c-47ff-8cb7-8609229d6caa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:28.703981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794e5fd-d14c-47ff-8cb7-8609229d6caa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:28.704258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794e5fd-d14c-47ff-8cb7-8609229d6caa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:28.776233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794e5fd-d14c-47ff-8cb7-8609229d6caa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:28.777427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794e5fd-d14c-47ff-8cb7-8609229d6caa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:28.803007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:28.817089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f41bddf-0749-4d9b-990c-13144af83225 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:28.885773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 335/672] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:28.885773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794e5fd-d14c-47ff-8cb7-8609229d6caa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:28.893507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eb911-a41c-40ec-b473-69a5121c56ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:28.895119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eb911-a41c-40ec-b473-69a5121c56ef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:28.895803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eb911-a41c-40ec-b473-69a5121c56ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:28.896482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4eb911-a41c-40ec-b473-69a5121c56ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:28.909713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4eb911-a41c-40ec-b473-69a5121c56ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:28.912900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4eb911-a41c-40ec-b473-69a5121c56ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-cd4eb911-a41c-40ec-b473-69a5121c56ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:28.933456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:28.934951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:28.935924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:28.935924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:28.935924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:28.935924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:28.935924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:28.935924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:28.935924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:28.935924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:28.944273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 338/673] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:28.946355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eb911-a41c-40ec-b473-69a5121c56ef None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:28.986024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794e5fd-d14c-47ff-8cb7-8609229d6caa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:29.008492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:29.013581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:29.026657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b61530-2ca3-4d22-80b9-848c13239977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:29.027315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b61530-2ca3-4d22-80b9-848c13239977 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.027597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b61530-2ca3-4d22-80b9-848c13239977 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.027874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b61530-2ca3-4d22-80b9-848c13239977 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.038423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b61530-2ca3-4d22-80b9-848c13239977 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:29.039876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b61530-2ca3-4d22-80b9-848c13239977 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-02b61530-2ca3-4d22-80b9-848c13239977 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:29.058011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:29.059003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:29.059941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:29.059941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:29.059941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:29.059941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:29.060727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 336/674] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:29.065976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b61530-2ca3-4d22-80b9-848c13239977 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:29.068067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4eb911-a41c-40ec-b473-69a5121c56ef None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:29.113791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3316af-3587-462a-b22f-33b19228603e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:29.113791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3316af-3587-462a-b22f-33b19228603e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.113791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3316af-3587-462a-b22f-33b19228603e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.113791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3316af-3587-462a-b22f-33b19228603e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.130491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3316af-3587-462a-b22f-33b19228603e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:29.132078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3316af-3587-462a-b22f-33b19228603e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-be3316af-3587-462a-b22f-33b19228603e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:29.147456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:29.148487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:29.149381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:29.149381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:29.149381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:29.149381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:29.153825 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 339/675] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:29.155332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c0b89-044a-4dcf-a875-01ba38d83c31 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.155621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c0b89-044a-4dcf-a875-01ba38d83c31 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.155898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4c0b89-044a-4dcf-a875-01ba38d83c31 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.157335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 340/676] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:29.158135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c0b89-044a-4dcf-a875-01ba38d83c31 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:29.183184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b61530-2ca3-4d22-80b9-848c13239977 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:29.219924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29d5ec-817d-465d-8e46-96b9366cb4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:29.220519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29d5ec-817d-465d-8e46-96b9366cb4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.220808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29d5ec-817d-465d-8e46-96b9366cb4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.221072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29d5ec-817d-465d-8e46-96b9366cb4f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.221945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29d5ec-817d-465d-8e46-96b9366cb4f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:29.222331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29d5ec-817d-465d-8e46-96b9366cb4f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:29.223829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29d5ec-817d-465d-8e46-96b9366cb4f0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:29.242566 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 337/677] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:29.243624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29d5ec-817d-465d-8e46-96b9366cb4f0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:29.265665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:29.272975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4c0b89-044a-4dcf-a875-01ba38d83c31 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:29.282786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:29.314242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301024ab-3d13-4043-9005-27aff49ee2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:29.314911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301024ab-3d13-4043-9005-27aff49ee2bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.315191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301024ab-3d13-4043-9005-27aff49ee2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.315460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301024ab-3d13-4043-9005-27aff49ee2bf None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.316243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301024ab-3d13-4043-9005-27aff49ee2bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:29.316544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301024ab-3d13-4043-9005-27aff49ee2bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:29.317992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301024ab-3d13-4043-9005-27aff49ee2bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:29.330895 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 341/678] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:29.332078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262734ff-53c8-4786-be65-120fa0e02131 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.332361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262734ff-53c8-4786-be65-120fa0e02131 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.332633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262734ff-53c8-4786-be65-120fa0e02131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.351672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-262734ff-53c8-4786-be65-120fa0e02131 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:29.352882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-262734ff-53c8-4786-be65-120fa0e02131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:29.361284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29d5ec-817d-465d-8e46-96b9366cb4f0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:29.399428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc824c8-269e-4ad4-ad3f-d00ef19e1396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:29.400059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc824c8-269e-4ad4-ad3f-d00ef19e1396 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.400339 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc824c8-269e-4ad4-ad3f-d00ef19e1396 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.400626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc824c8-269e-4ad4-ad3f-d00ef19e1396 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.401332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc824c8-269e-4ad4-ad3f-d00ef19e1396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:29.401634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc824c8-269e-4ad4-ad3f-d00ef19e1396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:29.403421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc824c8-269e-4ad4-ad3f-d00ef19e1396 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:29.420789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 338/679] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:29.420789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc824c8-269e-4ad4-ad3f-d00ef19e1396 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:29.434575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 342/680] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:29.436343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262734ff-53c8-4786-be65-120fa0e02131 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:29.534441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:29.540563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:29.540954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc824c8-269e-4ad4-ad3f-d00ef19e1396 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:29.554718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262734ff-53c8-4786-be65-120fa0e02131 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:29.579107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db105028-9da4-44c3-a54b-b4c11a760652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:29.579904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db105028-9da4-44c3-a54b-b4c11a760652 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.580581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db105028-9da4-44c3-a54b-b4c11a760652 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.580878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db105028-9da4-44c3-a54b-b4c11a760652 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.582151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db105028-9da4-44c3-a54b-b4c11a760652 None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=a3c6943686c843e588df14f7c6dffb8a) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:29.582469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db105028-9da4-44c3-a54b-b4c11a760652 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:29.585095 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db105028-9da4-44c3-a54b-b4c11a760652 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:29.592627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4fb3c-e6e0-47db-ae79-c48c48629004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:29.593298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4fb3c-e6e0-47db-ae79-c48c48629004 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.593577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4fb3c-e6e0-47db-ae79-c48c48629004 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.593899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4fb3c-e6e0-47db-ae79-c48c48629004 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.594381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 339/681] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:29 2022] GET /identity/v3/role_assignments?user.id=a3c6943686c843e588df14f7c6dffb8a&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:29.602468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4fb3c-e6e0-47db-ae79-c48c48629004 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:29.604073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4fb3c-e6e0-47db-ae79-c48c48629004 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:29.606820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db105028-9da4-44c3-a54b-b4c11a760652 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-aad4fb3c-e6e0-47db-ae79-c48c48629004 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:29.617922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:29.618975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:29.620020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:29.620020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:29.620020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:29.620020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:29.620020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:29.620020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:29.620020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:29.620020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:29.620020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:29.620020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:29.624264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 343/682] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:29.625349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4fb3c-e6e0-47db-ae79-c48c48629004 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:29.755869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4fb3c-e6e0-47db-ae79-c48c48629004 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:29.772873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db105028-9da4-44c3-a54b-b4c11a760652 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:29.801693 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6221b30-6380-4539-9d0d-b09124ce4ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:29.802300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6221b30-6380-4539-9d0d-b09124ce4ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.802585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6221b30-6380-4539-9d0d-b09124ce4ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.802842 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6221b30-6380-4539-9d0d-b09124ce4ca0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.815651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6221b30-6380-4539-9d0d-b09124ce4ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d6221b30-6380-4539-9d0d-b09124ce4ca0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:29.822762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:29.824106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 344/683] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:29.825283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6221b30-6380-4539-9d0d-b09124ce4ca0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:29.826831 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6498cfc-ff44-4a5e-ae27-30fc4539432b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:29.827321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6498cfc-ff44-4a5e-ae27-30fc4539432b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:29.827598 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6498cfc-ff44-4a5e-ae27-30fc4539432b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:29.827898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6498cfc-ff44-4a5e-ae27-30fc4539432b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:29.837407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6498cfc-ff44-4a5e-ae27-30fc4539432b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:29.838888 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6498cfc-ff44-4a5e-ae27-30fc4539432b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:29.840382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c6498cfc-ff44-4a5e-ae27-30fc4539432b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:29.854246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:29.855182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:29.856523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:29.856523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:29.856523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:29.856523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:29.856523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:29.856523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:29.856523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:29.856523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:29.856523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 340/684] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:29.856523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6498cfc-ff44-4a5e-ae27-30fc4539432b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:29.882415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:29.965918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6221b30-6380-4539-9d0d-b09124ce4ca0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:29.985535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6498cfc-ff44-4a5e-ae27-30fc4539432b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:30.010049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a0cdf-c02e-42ed-9f69-b8c6509c0772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:30.010944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a0cdf-c02e-42ed-9f69-b8c6509c0772 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.011454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a0cdf-c02e-42ed-9f69-b8c6509c0772 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.011954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a0cdf-c02e-42ed-9f69-b8c6509c0772 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.012957 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a0cdf-c02e-42ed-9f69-b8c6509c0772 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:30.013411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a0cdf-c02e-42ed-9f69-b8c6509c0772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:30.015388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a0cdf-c02e-42ed-9f69-b8c6509c0772 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:30.022891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d02e7-695c-4e85-9b83-9b6121f337e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:30.023550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d02e7-695c-4e85-9b83-9b6121f337e4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.023834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d02e7-695c-4e85-9b83-9b6121f337e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.024106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d02e7-695c-4e85-9b83-9b6121f337e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.024833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d02e7-695c-4e85-9b83-9b6121f337e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:30.025137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d02e7-695c-4e85-9b83-9b6121f337e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:30.026540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d02e7-695c-4e85-9b83-9b6121f337e4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:30.036037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 341/685] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:30.036996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d02e7-695c-4e85-9b83-9b6121f337e4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:30.039366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 345/686] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jul 26 07:12:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:30.049908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a0cdf-c02e-42ed-9f69-b8c6509c0772 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:30.125102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:30.130277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:30.150441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d02e7-695c-4e85-9b83-9b6121f337e4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:30.165409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a0cdf-c02e-42ed-9f69-b8c6509c0772 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:30.185795 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b15b98-9f62-4ac7-98c4-013af2a3f39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:30.186458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b15b98-9f62-4ac7-98c4-013af2a3f39f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.186715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b15b98-9f62-4ac7-98c4-013af2a3f39f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.186989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b15b98-9f62-4ac7-98c4-013af2a3f39f None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.188219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b15b98-9f62-4ac7-98c4-013af2a3f39f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:30.188535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b15b98-9f62-4ac7-98c4-013af2a3f39f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:30.190035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b15b98-9f62-4ac7-98c4-013af2a3f39f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:30.201639 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048012e-6b9c-48ef-af79-8d4bf2e097a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:30.202301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048012e-6b9c-48ef-af79-8d4bf2e097a3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.202580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048012e-6b9c-48ef-af79-8d4bf2e097a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.203018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 342/687] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:30.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b15b98-9f62-4ac7-98c4-013af2a3f39f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:30.207681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1048012e-6b9c-48ef-af79-8d4bf2e097a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.283792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048012e-6b9c-48ef-af79-8d4bf2e097a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:30.285302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1048012e-6b9c-48ef-af79-8d4bf2e097a3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:30.310325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b15b98-9f62-4ac7-98c4-013af2a3f39f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:30.347991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a458d-b9d8-4b9d-9906-586826610aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:30.348632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a458d-b9d8-4b9d-9906-586826610aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.349062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a458d-b9d8-4b9d-9906-586826610aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.349342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a458d-b9d8-4b9d-9906-586826610aa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.355105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:30.360704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a458d-b9d8-4b9d-9906-586826610aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:30.362220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a458d-b9d8-4b9d-9906-586826610aa2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:30.364561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-a92a458d-b9d8-4b9d-9906-586826610aa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:30.380833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:30.381796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:30.382682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:30.382682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:30.382682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:30.382682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:30.382682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 343/688] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:30.383614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bdb9b-91b2-45b4-ad6f-65bc2ece5c67 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.383614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bdb9b-91b2-45b4-ad6f-65bc2ece5c67 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.383773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bdb9b-91b2-45b4-ad6f-65bc2ece5c67 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.385142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 344/689] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:30.389776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bdb9b-91b2-45b4-ad6f-65bc2ece5c67 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:30.409517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 346/690] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:30.409517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048012e-6b9c-48ef-af79-8d4bf2e097a3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:30.507307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bdb9b-91b2-45b4-ad6f-65bc2ece5c67 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:30.520914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1048012e-6b9c-48ef-af79-8d4bf2e097a3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:30.543027 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f124c40-f82b-479c-89f8-ab5167a4ca62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:30.543906 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124c40-f82b-479c-89f8-ab5167a4ca62 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.544322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124c40-f82b-479c-89f8-ab5167a4ca62 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.544736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f124c40-f82b-479c-89f8-ab5167a4ca62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.545974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f124c40-f82b-479c-89f8-ab5167a4ca62 None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=1911678a39f749939dc8476617a884fa) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:30.546287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f124c40-f82b-479c-89f8-ab5167a4ca62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:30.547739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f124c40-f82b-479c-89f8-ab5167a4ca62 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:30.556927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eec80-365c-4e7c-839e-3ac9042b3db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:30.557566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eec80-365c-4e7c-839e-3ac9042b3db0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.557882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eec80-365c-4e7c-839e-3ac9042b3db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.558160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4eec80-365c-4e7c-839e-3ac9042b3db0 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.558896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4eec80-365c-4e7c-839e-3ac9042b3db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:30.559176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4eec80-365c-4e7c-839e-3ac9042b3db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:30.560600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4eec80-365c-4e7c-839e-3ac9042b3db0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:30.561943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 345/691] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:30 2022] GET /identity/v3/role_assignments?user.id=1911678a39f749939dc8476617a884fa&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:30.564641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4efe6-5bff-4a68-ab15-22e82a430f3d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.565275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4efe6-5bff-4a68-ab15-22e82a430f3d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.565844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a4efe6-5bff-4a68-ab15-22e82a430f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.581637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 347/692] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:30.582707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eec80-365c-4e7c-839e-3ac9042b3db0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:30.590812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-79a4efe6-5bff-4a68-ab15-22e82a430f3d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:30.592462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-79a4efe6-5bff-4a68-ab15-22e82a430f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:30.654243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:30.661361 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:30.679267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 346/693] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:30.680750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4efe6-5bff-4a68-ab15-22e82a430f3d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:30.752994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4eec80-365c-4e7c-839e-3ac9042b3db0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:30.798441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af4221-4bc8-415d-89cd-aff985b753fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:30.799091 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af4221-4bc8-415d-89cd-aff985b753fb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.800891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af4221-4bc8-415d-89cd-aff985b753fb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.800891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af4221-4bc8-415d-89cd-aff985b753fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.816005 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af4221-4bc8-415d-89cd-aff985b753fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:30.817504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9af4221-4bc8-415d-89cd-aff985b753fb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b9af4221-4bc8-415d-89cd-aff985b753fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:30.833384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:30.835062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:30.835975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:30.835975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:30.835975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:30.835975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:30.835975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 348/694] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:30.835975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af4221-4bc8-415d-89cd-aff985b753fb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:30.843260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a4efe6-5bff-4a68-ab15-22e82a430f3d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:30.898681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335f4f8-e715-4f25-b1f3-6c0d2996ccce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:30.899759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335f4f8-e715-4f25-b1f3-6c0d2996ccce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:30.900506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335f4f8-e715-4f25-b1f3-6c0d2996ccce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:30.901041 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335f4f8-e715-4f25-b1f3-6c0d2996ccce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:30.913637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335f4f8-e715-4f25-b1f3-6c0d2996ccce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:30.916043 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335f4f8-e715-4f25-b1f3-6c0d2996ccce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2335f4f8-e715-4f25-b1f3-6c0d2996ccce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:30.931264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:30.932266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:30.933209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:30.933209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:30.933209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:30.933209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:30.933209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:30.933209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:30.933209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:30.933209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:30.933209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:30.933209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:30.936353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 347/695] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jul 26 07:12:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:30.937719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335f4f8-e715-4f25-b1f3-6c0d2996ccce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:30.967754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:30.996862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af4221-4bc8-415d-89cd-aff985b753fb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:31.022221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:31.041099 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524059b-f141-4bf9-8d0f-fe48cbbb9f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:31.042399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524059b-f141-4bf9-8d0f-fe48cbbb9f62 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.042884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524059b-f141-4bf9-8d0f-fe48cbbb9f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.043373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524059b-f141-4bf9-8d0f-fe48cbbb9f62 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.044480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524059b-f141-4bf9-8d0f-fe48cbbb9f62 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:31.046799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524059b-f141-4bf9-8d0f-fe48cbbb9f62 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a524059b-f141-4bf9-8d0f-fe48cbbb9f62 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:31.061259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:31.062307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 07:12:31.063210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:31.071520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 349/696] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Jul 26 07:12:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:31.071682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524059b-f141-4bf9-8d0f-fe48cbbb9f62 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:31.109193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335f4f8-e715-4f25-b1f3-6c0d2996ccce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:31.146741 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30833298-03ff-4362-9446-72175c873089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:31.147373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30833298-03ff-4362-9446-72175c873089 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.147656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30833298-03ff-4362-9446-72175c873089 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.147933 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30833298-03ff-4362-9446-72175c873089 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.148747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30833298-03ff-4362-9446-72175c873089 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:31.149060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30833298-03ff-4362-9446-72175c873089 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:31.150667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30833298-03ff-4362-9446-72175c873089 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:31.164436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 348/697] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:31.165236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30833298-03ff-4362-9446-72175c873089 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:31.222681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524059b-f141-4bf9-8d0f-fe48cbbb9f62 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:31.262630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e222af-5316-413e-8181-2ef19b4b0c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:31.263259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e222af-5316-413e-8181-2ef19b4b0c89 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.263539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e222af-5316-413e-8181-2ef19b4b0c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.263809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e222af-5316-413e-8181-2ef19b4b0c89 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.264609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e222af-5316-413e-8181-2ef19b4b0c89 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:31.264914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e222af-5316-413e-8181-2ef19b4b0c89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:31.266406 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e222af-5316-413e-8181-2ef19b4b0c89 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:31.281112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:31.283569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30833298-03ff-4362-9446-72175c873089 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:31.287499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 350/698] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jul 26 07:12:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:31.288457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e222af-5316-413e-8181-2ef19b4b0c89 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:31.296377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:31.324178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5a230-2660-44eb-975a-17410d04c58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:31.325219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5a230-2660-44eb-975a-17410d04c58e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.325725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5a230-2660-44eb-975a-17410d04c58e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.326243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5a230-2660-44eb-975a-17410d04c58e None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.327143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5a230-2660-44eb-975a-17410d04c58e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:31.327750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5a230-2660-44eb-975a-17410d04c58e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:31.329357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5a230-2660-44eb-975a-17410d04c58e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:31.339843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 349/699] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jul 26 07:12:31 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:31.341142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5a230-2660-44eb-975a-17410d04c58e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:31.395503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e222af-5316-413e-8181-2ef19b4b0c89 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:31.431542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d51c6f-1a1d-463c-8a74-756d47d0fccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:31.432106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d51c6f-1a1d-463c-8a74-756d47d0fccd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.432384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d51c6f-1a1d-463c-8a74-756d47d0fccd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.432655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d51c6f-1a1d-463c-8a74-756d47d0fccd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.440486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d51c6f-1a1d-463c-8a74-756d47d0fccd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:31.442189 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d51c6f-1a1d-463c-8a74-756d47d0fccd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:31.451421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5a230-2660-44eb-975a-17410d04c58e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-03d51c6f-1a1d-463c-8a74-756d47d0fccd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:31.454800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:31.455706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:31.456665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:31.456665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:31.456665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:31.456665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:31.456665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:31.456665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:31.456665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:31.456665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:31.460569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 351/700] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:31.461442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d51c6f-1a1d-463c-8a74-756d47d0fccd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:31.491587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c14267-c1fe-4415-be4c-ce1203b541b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:31.492605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c14267-c1fe-4415-be4c-ce1203b541b5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.493113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c14267-c1fe-4415-be4c-ce1203b541b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.493608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c14267-c1fe-4415-be4c-ce1203b541b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.525774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:31.537440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:31.565294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c14267-c1fe-4415-be4c-ce1203b541b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:31.567174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c14267-c1fe-4415-be4c-ce1203b541b5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:31.572056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d51c6f-1a1d-463c-8a74-756d47d0fccd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:31.609092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebcb18-e7b5-4de2-b8bd-08d13f0b6e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:31.609729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebcb18-e7b5-4de2-b8bd-08d13f0b6e15 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.610064 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebcb18-e7b5-4de2-b8bd-08d13f0b6e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.610349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ebcb18-e7b5-4de2-b8bd-08d13f0b6e15 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.611021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebcb18-e7b5-4de2-b8bd-08d13f0b6e15 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:31.611319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebcb18-e7b5-4de2-b8bd-08d13f0b6e15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:31.612944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ebcb18-e7b5-4de2-b8bd-08d13f0b6e15 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:31.625921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 352/701] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jul 26 07:12:31 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:31.625921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebcb18-e7b5-4de2-b8bd-08d13f0b6e15 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:31.716798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 350/702] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:31.722409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c14267-c1fe-4415-be4c-ce1203b541b5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:31.736022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ebcb18-e7b5-4de2-b8bd-08d13f0b6e15 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:31.762472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:31.777298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e000aa-e389-48fc-b132-b97d73f9ecb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:31.778080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e000aa-e389-48fc-b132-b97d73f9ecb6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.778359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e000aa-e389-48fc-b132-b97d73f9ecb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.778633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e000aa-e389-48fc-b132-b97d73f9ecb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.779411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e000aa-e389-48fc-b132-b97d73f9ecb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:31.779716 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e000aa-e389-48fc-b132-b97d73f9ecb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:31.781234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e000aa-e389-48fc-b132-b97d73f9ecb6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:31.796594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 353/703] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:31.802897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678bba5-f6cc-4d6d-970c-8f1ad0ba5805 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.803550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678bba5-f6cc-4d6d-970c-8f1ad0ba5805 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.803550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4678bba5-f6cc-4d6d-970c-8f1ad0ba5805 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.805454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 354/704] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:31.807230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678bba5-f6cc-4d6d-970c-8f1ad0ba5805 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:31.826143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:31.854067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c14267-c1fe-4415-be4c-ce1203b541b5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:31.900797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5200f4cc-5407-45ea-84f7-0c4123571959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:31.902051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200f4cc-5407-45ea-84f7-0c4123571959 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.902577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200f4cc-5407-45ea-84f7-0c4123571959 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.903129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5200f4cc-5407-45ea-84f7-0c4123571959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.916922 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200f4cc-5407-45ea-84f7-0c4123571959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:31.920702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5200f4cc-5407-45ea-84f7-0c4123571959 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5200f4cc-5407-45ea-84f7-0c4123571959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:31.935194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:31.936145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:31.937019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:31.937019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:31.937019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:31.937019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:31.937019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 351/705] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:31.937700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb4d6d-dfac-4294-b8a4-dc49da767907 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:31.937915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb4d6d-dfac-4294-b8a4-dc49da767907 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:31.938183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb4d6d-dfac-4294-b8a4-dc49da767907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:31.948822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4678bba5-f6cc-4d6d-970c-8f1ad0ba5805 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:31.958885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5eeb4d6d-dfac-4294-b8a4-dc49da767907 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:31.960226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5eeb4d6d-dfac-4294-b8a4-dc49da767907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:32.000346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45d2f5-570b-4b45-a250-75f1bf518573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:32.001362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45d2f5-570b-4b45-a250-75f1bf518573 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:32.001862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45d2f5-570b-4b45-a250-75f1bf518573 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:32.002204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45d2f5-570b-4b45-a250-75f1bf518573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:32.003131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45d2f5-570b-4b45-a250-75f1bf518573 None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=64257d34b2df4af1a2016d56be8d44a8) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:32.003905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45d2f5-570b-4b45-a250-75f1bf518573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:32.006865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45d2f5-570b-4b45-a250-75f1bf518573 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:32.016186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 355/706] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:31 2022] GET /identity/v3/role_assignments?user.id=64257d34b2df4af1a2016d56be8d44a8&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:32.018813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45d2f5-570b-4b45-a250-75f1bf518573 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:32.068000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 352/707] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:32.099122 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb4d6d-dfac-4294-b8a4-dc49da767907 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:32.117529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:32.180181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45d2f5-570b-4b45-a250-75f1bf518573 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:32.227797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb671d9-f9f4-4648-ae8e-7d9964ef3d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:32.229682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb671d9-f9f4-4648-ae8e-7d9964ef3d7e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:32.230246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb671d9-f9f4-4648-ae8e-7d9964ef3d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:32.234252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb671d9-f9f4-4648-ae8e-7d9964ef3d7e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:32.235417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb671d9-f9f4-4648-ae8e-7d9964ef3d7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:32.235731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb671d9-f9f4-4648-ae8e-7d9964ef3d7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:32.237173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb671d9-f9f4-4648-ae8e-7d9964ef3d7e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:32.240877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:32.252551 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 356/708] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:32.259809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb671d9-f9f4-4648-ae8e-7d9964ef3d7e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:32.320767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeb4d6d-dfac-4294-b8a4-dc49da767907 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:32.387322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b252348-73fd-4e79-af48-7ef02eb4d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:32.388341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b252348-73fd-4e79-af48-7ef02eb4d79c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:32.388992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b252348-73fd-4e79-af48-7ef02eb4d79c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:32.390780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b252348-73fd-4e79-af48-7ef02eb4d79c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:32.412005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b252348-73fd-4e79-af48-7ef02eb4d79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:32.414455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b252348-73fd-4e79-af48-7ef02eb4d79c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1b252348-73fd-4e79-af48-7ef02eb4d79c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:32.452628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:32.454639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:32.456005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:32.456005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:32.456005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:32.456005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:32.456576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb671d9-f9f4-4648-ae8e-7d9964ef3d7e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:32.458650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 353/709] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:32.465857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b252348-73fd-4e79-af48-7ef02eb4d79c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:32.472414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:32.536800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe12ee-e435-41fa-ac21-202245700a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:32.537854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe12ee-e435-41fa-ac21-202245700a98 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:32.538154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe12ee-e435-41fa-ac21-202245700a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:32.538565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbe12ee-e435-41fa-ac21-202245700a98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:32.553723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe12ee-e435-41fa-ac21-202245700a98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:32.556637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbe12ee-e435-41fa-ac21-202245700a98 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8cbe12ee-e435-41fa-ac21-202245700a98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:32.576567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:32.577585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:32.578538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:32.578538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:32.578538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:32.578538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:32.578538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:32.578538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:32.578538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:32.578538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:32.579613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 357/710] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:32.585076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe12ee-e435-41fa-ac21-202245700a98 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:32.613894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b252348-73fd-4e79-af48-7ef02eb4d79c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:32.626659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:32.667865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b3d29-62f7-4a00-8693-017f7da5794e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:32.668514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b3d29-62f7-4a00-8693-017f7da5794e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:32.668866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b3d29-62f7-4a00-8693-017f7da5794e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:32.669302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b3d29-62f7-4a00-8693-017f7da5794e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:32.674384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b3d29-62f7-4a00-8693-017f7da5794e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:32.674913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b3d29-62f7-4a00-8693-017f7da5794e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:32.678451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b3d29-62f7-4a00-8693-017f7da5794e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:32.706970 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 354/711] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:32.711296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b3d29-62f7-4a00-8693-017f7da5794e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:32.809874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbe12ee-e435-41fa-ac21-202245700a98 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:32.890612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db0acb-7a8d-4f1d-81ce-550d82364aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:32.891308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db0acb-7a8d-4f1d-81ce-550d82364aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:32.891587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db0acb-7a8d-4f1d-81ce-550d82364aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:32.891883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7db0acb-7a8d-4f1d-81ce-550d82364aa5 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:32.892740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db0acb-7a8d-4f1d-81ce-550d82364aa5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:32.893033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db0acb-7a8d-4f1d-81ce-550d82364aa5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:32.894655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7db0acb-7a8d-4f1d-81ce-550d82364aa5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:32.926174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 358/712] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:32.935230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b3d29-62f7-4a00-8693-017f7da5794e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:32.944474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db0acb-7a8d-4f1d-81ce-550d82364aa5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:33.002037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:33.004674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1447dab-9d13-4b15-816e-7d4dc724ac21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:33.007636 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1447dab-9d13-4b15-816e-7d4dc724ac21 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:33.008325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1447dab-9d13-4b15-816e-7d4dc724ac21 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:33.008994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1447dab-9d13-4b15-816e-7d4dc724ac21 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:33.010996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1447dab-9d13-4b15-816e-7d4dc724ac21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:33.013912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1447dab-9d13-4b15-816e-7d4dc724ac21 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:33.057868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 355/713] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:12:32 2022] POST /identity/v3/users => generated 254 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:12:33.057868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfa17d-219e-42fe-8e43-b2c73a7b32fa None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:33.057868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfa17d-219e-42fe-8e43-b2c73a7b32fa None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:33.057868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bfa17d-219e-42fe-8e43-b2c73a7b32fa None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:33.066466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 356/714] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:33.073979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c44ef9-935b-4740-a227-31c65aae75a8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:33.073979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c44ef9-935b-4740-a227-31c65aae75a8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:33.073979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c44ef9-935b-4740-a227-31c65aae75a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:33.106085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-49c44ef9-935b-4740-a227-31c65aae75a8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:33.113904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49c44ef9-935b-4740-a227-31c65aae75a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:33.133957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:33.192508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7db0acb-7a8d-4f1d-81ce-550d82364aa5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:33.217908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 357/715] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 26 07:12:33.247422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c44ef9-935b-4740-a227-31c65aae75a8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:33.257939 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be6722-c288-402c-bcd8-3da607888d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:33.258601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be6722-c288-402c-bcd8-3da607888d84 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:33.258924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be6722-c288-402c-bcd8-3da607888d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:33.259195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be6722-c288-402c-bcd8-3da607888d84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:33.260105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be6722-c288-402c-bcd8-3da607888d84 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=6d50427014064f099af87932c77a849a) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:33.260417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be6722-c288-402c-bcd8-3da607888d84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:33.262020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20be6722-c288-402c-bcd8-3da607888d84 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:33.272072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 359/716] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:32 2022] GET /identity/v3/role_assignments?user.id=6d50427014064f099af87932c77a849a&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 259 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:33.380211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c44ef9-935b-4740-a227-31c65aae75a8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:33.424127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b96e8-62f2-4d32-931f-b5a274cf0cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:33.424815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b96e8-62f2-4d32-931f-b5a274cf0cab None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:33.425090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b96e8-62f2-4d32-931f-b5a274cf0cab None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:33.425435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1b96e8-62f2-4d32-931f-b5a274cf0cab None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:33.426209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b96e8-62f2-4d32-931f-b5a274cf0cab None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:33.427822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1b96e8-62f2-4d32-931f-b5a274cf0cab None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-fb1b96e8-62f2-4d32-931f-b5a274cf0cab None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:33.444953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:33.446100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:33.446925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 358/717] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:33 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:33.454951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be6722-c288-402c-bcd8-3da607888d84 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:33.572185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:33.581292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be6722-c288-402c-bcd8-3da607888d84 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:33.617921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44f18c-630a-4e52-a0a3-0d35e39ed9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:33.618619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44f18c-630a-4e52-a0a3-0d35e39ed9fa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:33.618898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44f18c-630a-4e52-a0a3-0d35e39ed9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:33.619171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44f18c-630a-4e52-a0a3-0d35e39ed9fa None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:33.619865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44f18c-630a-4e52-a0a3-0d35e39ed9fa None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:33.620165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44f18c-630a-4e52-a0a3-0d35e39ed9fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:33.621700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44f18c-630a-4e52-a0a3-0d35e39ed9fa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:33.627002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 360/718] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:12:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:33.633144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b96e8-62f2-4d32-931f-b5a274cf0cab None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:33.660210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:33.772503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1b96e8-62f2-4d32-931f-b5a274cf0cab None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:33.809655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212a714a-43d7-44c2-8dac-ec1a26b07d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:33.810330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a714a-43d7-44c2-8dac-ec1a26b07d22 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:33.810607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a714a-43d7-44c2-8dac-ec1a26b07d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:33.810879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212a714a-43d7-44c2-8dac-ec1a26b07d22 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:33.811574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a714a-43d7-44c2-8dac-ec1a26b07d22 None admin] RBAC: Adding query filter params (type=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:33.811878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a714a-43d7-44c2-8dac-ec1a26b07d22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:33.813263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212a714a-43d7-44c2-8dac-ec1a26b07d22 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:33.824157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 359/719] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:12:33 2022] GET /identity/v3/services?type=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:34.197335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1b5d-438c-4ea1-b202-455a84c91185 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:34.197924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1b5d-438c-4ea1-b202-455a84c91185 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:34.197924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcd1b5d-438c-4ea1-b202-455a84c91185 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:34.199769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 361/720] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:34.207107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d153a0f-004c-4495-b8c1-0fcf16ce04ae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:34.207645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d153a0f-004c-4495-b8c1-0fcf16ce04ae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:34.207645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d153a0f-004c-4495-b8c1-0fcf16ce04ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:34.361072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4d153a0f-004c-4495-b8c1-0fcf16ce04ae None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:34.362319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4d153a0f-004c-4495-b8c1-0fcf16ce04ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:34.475948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 360/721] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:34.515336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd1b5d-438c-4ea1-b202-455a84c91185 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:34.635548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcd1b5d-438c-4ea1-b202-455a84c91185 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:34.671995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751eda0-5115-420e-b9cb-7d31aac5bd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:34.672705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751eda0-5115-420e-b9cb-7d31aac5bd16 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:34.672986 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751eda0-5115-420e-b9cb-7d31aac5bd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:34.673267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751eda0-5115-420e-b9cb-7d31aac5bd16 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:34.674027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751eda0-5115-420e-b9cb-7d31aac5bd16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:34.675724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751eda0-5115-420e-b9cb-7d31aac5bd16 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:34.694960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 362/722] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:12:34 2022] POST /identity/v3/services => generated 251 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:35.073853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935b33f-b82d-4438-ad7a-8f0dead87448 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:35.074564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935b33f-b82d-4438-ad7a-8f0dead87448 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:35.075096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935b33f-b82d-4438-ad7a-8f0dead87448 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:35.076936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 361/723] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:35.083839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a126565-481d-4e64-a88f-a544e3e996d2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:35.084311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a126565-481d-4e64-a88f-a544e3e996d2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:35.084777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a126565-481d-4e64-a88f-a544e3e996d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:35.110441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7a126565-481d-4e64-a88f-a544e3e996d2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:35.112026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7a126565-481d-4e64-a88f-a544e3e996d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:35.125050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:35.199597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 363/724] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:35.222277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935b33f-b82d-4438-ad7a-8f0dead87448 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:35.242426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:35.357825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935b33f-b82d-4438-ad7a-8f0dead87448 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:35.394156 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb2e6d-e2ec-499b-975e-5f242d066a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:35.394822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb2e6d-e2ec-499b-975e-5f242d066a14 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:35.395106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb2e6d-e2ec-499b-975e-5f242d066a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:35.395383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fb2e6d-e2ec-499b-975e-5f242d066a14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:35.406158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fb2e6d-e2ec-499b-975e-5f242d066a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e6fb2e6d-e2ec-499b-975e-5f242d066a14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:35.411272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:35.412261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:35.413138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:35.417527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 362/725] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:35.419071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a126565-481d-4e64-a88f-a544e3e996d2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:35.571011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a126565-481d-4e64-a88f-a544e3e996d2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:35.623424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e782b-8f7b-48f4-8c4c-70e0a4edba45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:35.624443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e782b-8f7b-48f4-8c4c-70e0a4edba45 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:35.624982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e782b-8f7b-48f4-8c4c-70e0a4edba45 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:35.625464 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705e782b-8f7b-48f4-8c4c-70e0a4edba45 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:35.626419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e782b-8f7b-48f4-8c4c-70e0a4edba45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:35.626960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e782b-8f7b-48f4-8c4c-70e0a4edba45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:35.628583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705e782b-8f7b-48f4-8c4c-70e0a4edba45 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:35.641536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 364/726] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:35.647497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb2e6d-e2ec-499b-975e-5f242d066a14 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:35.743446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:35.775558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fb2e6d-e2ec-499b-975e-5f242d066a14 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:35.810020 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b1d6-7b60-496d-907c-34e540088d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:35.810677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b1d6-7b60-496d-907c-34e540088d7e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:35.810961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b1d6-7b60-496d-907c-34e540088d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:35.811223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b1d6-7b60-496d-907c-34e540088d7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:35.822802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48b1d6-7b60-496d-907c-34e540088d7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:35.824286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48b1d6-7b60-496d-907c-34e540088d7e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cf48b1d6-7b60-496d-907c-34e540088d7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:35.839954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:35.840931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:35.841862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:35.841862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:35.841862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:35.841862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:35.849857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 363/727] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:35.852268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e782b-8f7b-48f4-8c4c-70e0a4edba45 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:35.856359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:35.981338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705e782b-8f7b-48f4-8c4c-70e0a4edba45 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:36.034485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b20bc12-ef8b-4ac1-b051-b52664112c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:36.035196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20bc12-ef8b-4ac1-b051-b52664112c3f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:36.035483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20bc12-ef8b-4ac1-b051-b52664112c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:36.035761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b20bc12-ef8b-4ac1-b051-b52664112c3f None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:36.036470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20bc12-ef8b-4ac1-b051-b52664112c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:36.036776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20bc12-ef8b-4ac1-b051-b52664112c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:36.038214 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b20bc12-ef8b-4ac1-b051-b52664112c3f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:36.052196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 365/728] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:36.057065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b1d6-7b60-496d-907c-34e540088d7e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:36.126616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec09d12-8555-4f09-8fca-21c93c947749 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:36.126995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec09d12-8555-4f09-8fca-21c93c947749 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:36.127209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec09d12-8555-4f09-8fca-21c93c947749 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:36.128787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 366/729] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:36.134496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141982d3-5510-414e-8eec-f584bf27a682 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:36.135814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141982d3-5510-414e-8eec-f584bf27a682 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:36.136165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141982d3-5510-414e-8eec-f584bf27a682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:36.183968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-141982d3-5510-414e-8eec-f584bf27a682 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:36.189015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b1d6-7b60-496d-907c-34e540088d7e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:36.191394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-141982d3-5510-414e-8eec-f584bf27a682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:36.241097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:36.268158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707f4de-44cc-479e-b5df-8c7fdc0bf950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:36.268865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707f4de-44cc-479e-b5df-8c7fdc0bf950 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:36.269313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707f4de-44cc-479e-b5df-8c7fdc0bf950 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:36.269607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707f4de-44cc-479e-b5df-8c7fdc0bf950 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:36.290017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707f4de-44cc-479e-b5df-8c7fdc0bf950 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:36.291488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5707f4de-44cc-479e-b5df-8c7fdc0bf950 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5707f4de-44cc-479e-b5df-8c7fdc0bf950 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:36.316732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:36.317988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:36.318961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:36.318961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:36.318961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:36.318961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:36.318961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:36.318961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:36.318961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:36.318961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:36.318961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:36.318961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:36.326392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 364/730] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:36.327116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:36.327564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707f4de-44cc-479e-b5df-8c7fdc0bf950 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:36.340667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 367/731] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:36.341802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29dcad-181c-4930-8d1e-c332b2dbc3c1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:36.342097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29dcad-181c-4930-8d1e-c332b2dbc3c1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:36.342368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b29dcad-181c-4930-8d1e-c332b2dbc3c1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:36.344161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 368/732] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:36.354852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a766cc2-9c03-4243-845f-bf5d6307c3f2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:36.355278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a766cc2-9c03-4243-845f-bf5d6307c3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:36.355652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a766cc2-9c03-4243-845f-bf5d6307c3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:36.387007 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9a766cc2-9c03-4243-845f-bf5d6307c3f2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:36.388724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a766cc2-9c03-4243-845f-bf5d6307c3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:36.494179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 369/733] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:36 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:36.495455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a766cc2-9c03-4243-845f-bf5d6307c3f2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:36.530254 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5707f4de-44cc-479e-b5df-8c7fdc0bf950 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:36.588131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade1dc4-0589-46ee-b42e-cb12046c50f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:36.589288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade1dc4-0589-46ee-b42e-cb12046c50f5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:36.589808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade1dc4-0589-46ee-b42e-cb12046c50f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:36.590298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dade1dc4-0589-46ee-b42e-cb12046c50f5 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:36.591872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade1dc4-0589-46ee-b42e-cb12046c50f5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:36.592367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade1dc4-0589-46ee-b42e-cb12046c50f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:36.595153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dade1dc4-0589-46ee-b42e-cb12046c50f5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:36.625471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a766cc2-9c03-4243-845f-bf5d6307c3f2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:36.629144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 365/734] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jul 26 07:12:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:36.630120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade1dc4-0589-46ee-b42e-cb12046c50f5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:36.653349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:36.667446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9085b9-8c45-4ef7-8e9c-11c15f6f3dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:36.668512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9085b9-8c45-4ef7-8e9c-11c15f6f3dae None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:36.669113 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9085b9-8c45-4ef7-8e9c-11c15f6f3dae None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:36.669585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9085b9-8c45-4ef7-8e9c-11c15f6f3dae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:36.682028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9085b9-8c45-4ef7-8e9c-11c15f6f3dae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fd9085b9-8c45-4ef7-8e9c-11c15f6f3dae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:36.687626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:36.688563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:36.689403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:36.697138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 370/735] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:36.698414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9085b9-8c45-4ef7-8e9c-11c15f6f3dae None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:36.705905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:36.795075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dade1dc4-0589-46ee-b42e-cb12046c50f5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:36.842399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4975fd0-18ea-4d8a-ba7a-2aa7fe2ffda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:36.843182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4975fd0-18ea-4d8a-ba7a-2aa7fe2ffda6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:36.843464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4975fd0-18ea-4d8a-ba7a-2aa7fe2ffda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:36.843737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4975fd0-18ea-4d8a-ba7a-2aa7fe2ffda6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:36.849881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9085b9-8c45-4ef7-8e9c-11c15f6f3dae None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:36.856997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4975fd0-18ea-4d8a-ba7a-2aa7fe2ffda6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f4975fd0-18ea-4d8a-ba7a-2aa7fe2ffda6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:36.863825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:36.864817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:36.865675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 366/736] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:36.867374 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4975fd0-18ea-4d8a-ba7a-2aa7fe2ffda6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:36.903663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61593c2-7f15-4279-927e-56bf48355ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:36.904692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61593c2-7f15-4279-927e-56bf48355ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:36.905146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61593c2-7f15-4279-927e-56bf48355ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:36.905439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61593c2-7f15-4279-927e-56bf48355ea8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:36.964526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:36.977563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:36.980642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4975fd0-18ea-4d8a-ba7a-2aa7fe2ffda6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:36.982087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61593c2-7f15-4279-927e-56bf48355ea8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:36.984513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61593c2-7f15-4279-927e-56bf48355ea8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:37.018818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a4b6e2-69d9-44e8-a019-c84940776980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:37.019580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4b6e2-69d9-44e8-a019-c84940776980 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.019860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4b6e2-69d9-44e8-a019-c84940776980 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.020133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a4b6e2-69d9-44e8-a019-c84940776980 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.020854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4b6e2-69d9-44e8-a019-c84940776980 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:37.021153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4b6e2-69d9-44e8-a019-c84940776980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:37.022570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a4b6e2-69d9-44e8-a019-c84940776980 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:37.031248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 367/737] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:37.032525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ed84-f2e5-419a-8737-2b752973a68c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.032810 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ed84-f2e5-419a-8737-2b752973a68c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.033079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79ed84-f2e5-419a-8737-2b752973a68c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.036857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 368/738] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:37.036857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ed84-f2e5-419a-8737-2b752973a68c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:37.101883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 371/739] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:36 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 26 07:12:37.103072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61593c2-7f15-4279-927e-56bf48355ea8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:37.155561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79ed84-f2e5-419a-8737-2b752973a68c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:37.193124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f223ed6-d3d1-4cda-a130-060c4fe263ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:37.193797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f223ed6-d3d1-4cda-a130-060c4fe263ef None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.194079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f223ed6-d3d1-4cda-a130-060c4fe263ef None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.194349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f223ed6-d3d1-4cda-a130-060c4fe263ef None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.195089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f223ed6-d3d1-4cda-a130-060c4fe263ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:37.195391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f223ed6-d3d1-4cda-a130-060c4fe263ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:37.196827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f223ed6-d3d1-4cda-a130-060c4fe263ef None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:37.206170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:37.207010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 369/740] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:37.211319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecdbe6-5476-4f83-a705-f58f0d62e92e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.211790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecdbe6-5476-4f83-a705-f58f0d62e92e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.213126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecdbe6-5476-4f83-a705-f58f0d62e92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.227555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61593c2-7f15-4279-927e-56bf48355ea8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:37.237079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b6ecdbe6-5476-4f83-a705-f58f0d62e92e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:37.238412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b6ecdbe6-5476-4f83-a705-f58f0d62e92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:37.263118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:37.268084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4eb68d-1572-4171-969d-d20833b0347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:37.268084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4eb68d-1572-4171-969d-d20833b0347c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.268084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4eb68d-1572-4171-969d-d20833b0347c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.268435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4eb68d-1572-4171-969d-d20833b0347c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.278997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4eb68d-1572-4171-969d-d20833b0347c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:37.280563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4eb68d-1572-4171-969d-d20833b0347c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ed4eb68d-1572-4171-969d-d20833b0347c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:37.301728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:37.302897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:37.304002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:37.304002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:37.304002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:37.304002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:37.304002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 372/741] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:37.304002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4eb68d-1572-4171-969d-d20833b0347c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:37.333955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 370/742] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 26 07:12:37.333955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecdbe6-5476-4f83-a705-f58f0d62e92e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:37.410669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4eb68d-1572-4171-969d-d20833b0347c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:37.441491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecdbe6-5476-4f83-a705-f58f0d62e92e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:37.447043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6427b8fa-be2f-4c80-805a-e596f67491bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:37.447695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427b8fa-be2f-4c80-805a-e596f67491bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.447973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427b8fa-be2f-4c80-805a-e596f67491bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.448241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6427b8fa-be2f-4c80-805a-e596f67491bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.451751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:37.461258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427b8fa-be2f-4c80-805a-e596f67491bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:37.462864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6427b8fa-be2f-4c80-805a-e596f67491bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:37.476083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f528c-b088-4ccb-bbde-76862be7050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:37.476741 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f528c-b088-4ccb-bbde-76862be7050c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.477024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f528c-b088-4ccb-bbde-76862be7050c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6427b8fa-be2f-4c80-805a-e596f67491bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:37.477161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:37.478278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:37.479322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:37.479322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:37.479322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:37.479322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:37.479800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f528c-b088-4ccb-bbde-76862be7050c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.479871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 373/743] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:37.479871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae9812-0718-47d9-8e1b-683ef7e66568 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.479871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae9812-0718-47d9-8e1b-683ef7e66568 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.480051 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae9812-0718-47d9-8e1b-683ef7e66568 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.481411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 374/744] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:37.485776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae9812-0718-47d9-8e1b-683ef7e66568 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:37.493953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f528c-b088-4ccb-bbde-76862be7050c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:37.495787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f528c-b088-4ccb-bbde-76862be7050c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3e8f528c-b088-4ccb-bbde-76862be7050c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:37.510901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:37.511935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:37.512834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:37.512834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:37.512834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:37.512834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:37.512834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 371/745] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:37.513110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f528c-b088-4ccb-bbde-76862be7050c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:37.521995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:37.593581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae9812-0718-47d9-8e1b-683ef7e66568 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:37.623580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f528c-b088-4ccb-bbde-76862be7050c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:37.630643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac065-e289-4877-9b27-50a17d25cda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:37.631280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac065-e289-4877-9b27-50a17d25cda5 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.631562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac065-e289-4877-9b27-50a17d25cda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.631841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cac065-e289-4877-9b27-50a17d25cda5 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.632538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac065-e289-4877-9b27-50a17d25cda5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:37.632848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac065-e289-4877-9b27-50a17d25cda5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:37.634275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cac065-e289-4877-9b27-50a17d25cda5 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:37.646607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 375/746] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:37.647486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac065-e289-4877-9b27-50a17d25cda5 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:37.664078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d62438-3ffe-419e-b7fe-8f61bea97f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:37.665012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d62438-3ffe-419e-b7fe-8f61bea97f90 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.665293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d62438-3ffe-419e-b7fe-8f61bea97f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.665568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d62438-3ffe-419e-b7fe-8f61bea97f90 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.666305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d62438-3ffe-419e-b7fe-8f61bea97f90 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:37.667643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d62438-3ffe-419e-b7fe-8f61bea97f90 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:37.684433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 372/747] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:12:37 2022] POST /identity/v3/services => generated 240 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:37.685730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcecdc5-f104-494d-99b7-d54348821f7a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.686076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcecdc5-f104-494d-99b7-d54348821f7a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.686358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcecdc5-f104-494d-99b7-d54348821f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.704553 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:37.706290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8fcecdc5-f104-494d-99b7-d54348821f7a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:37.707694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8fcecdc5-f104-494d-99b7-d54348821f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:37.774043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cac065-e289-4877-9b27-50a17d25cda5 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:37.783101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:37.813780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 373/748] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:37.813780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcecdc5-f104-494d-99b7-d54348821f7a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:37.833841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822acbe-1e47-4f58-891e-684498fc817e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:37.834556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822acbe-1e47-4f58-891e-684498fc817e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.834779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822acbe-1e47-4f58-891e-684498fc817e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.835055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822acbe-1e47-4f58-891e-684498fc817e None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.836122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822acbe-1e47-4f58-891e-684498fc817e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:37.836122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822acbe-1e47-4f58-891e-684498fc817e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:37.837487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822acbe-1e47-4f58-891e-684498fc817e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:37.850067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 376/749] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:37.850851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822acbe-1e47-4f58-891e-684498fc817e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:37.925959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcecdc5-f104-494d-99b7-d54348821f7a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:37.962471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9d364-ac2e-4e2f-80f8-552a378e477b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:37.963286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9d364-ac2e-4e2f-80f8-552a378e477b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:37.963707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9d364-ac2e-4e2f-80f8-552a378e477b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:37.963975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9d364-ac2e-4e2f-80f8-552a378e477b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:37.964682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9d364-ac2e-4e2f-80f8-552a378e477b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:37.964985 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9d364-ac2e-4e2f-80f8-552a378e477b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:37.966086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822acbe-1e47-4f58-891e-684498fc817e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:37.966396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9d364-ac2e-4e2f-80f8-552a378e477b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:37.977083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:37.985349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 374/750] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:37.985349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9d364-ac2e-4e2f-80f8-552a378e477b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:38.009105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc59f40-a42a-42c2-a02e-daae4a601ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:38.009795 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc59f40-a42a-42c2-a02e-daae4a601ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:38.010067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc59f40-a42a-42c2-a02e-daae4a601ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:38.010331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc59f40-a42a-42c2-a02e-daae4a601ff6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:38.019284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc59f40-a42a-42c2-a02e-daae4a601ff6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:38.020795 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc59f40-a42a-42c2-a02e-daae4a601ff6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2bc59f40-a42a-42c2-a02e-daae4a601ff6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:38.033621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:38.034757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:38.035728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 377/751] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jul 26 07:12:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:38.036322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc59f40-a42a-42c2-a02e-daae4a601ff6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:38.055085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:38.112575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9d364-ac2e-4e2f-80f8-552a378e477b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:38.161407 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc59f40-a42a-42c2-a02e-daae4a601ff6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:38.166625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351de52-7cb3-4892-b637-1434b3adeb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:38.167672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351de52-7cb3-4892-b637-1434b3adeb75 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:38.168128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351de52-7cb3-4892-b637-1434b3adeb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:38.168576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351de52-7cb3-4892-b637-1434b3adeb75 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:38.169676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351de52-7cb3-4892-b637-1434b3adeb75 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:38.172191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351de52-7cb3-4892-b637-1434b3adeb75 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2351de52-7cb3-4892-b637-1434b3adeb75 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:38.185046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:38.186175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:38.187884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 375/752] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Jul 26 07:12:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:38.188727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351de52-7cb3-4892-b637-1434b3adeb75 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:38.207723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea34e1b-4a9b-4db0-b018-8d17ecbdb08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:38.208719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea34e1b-4a9b-4db0-b018-8d17ecbdb08c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:38.208993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea34e1b-4a9b-4db0-b018-8d17ecbdb08c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:38.209267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea34e1b-4a9b-4db0-b018-8d17ecbdb08c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:38.216718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea34e1b-4a9b-4db0-b018-8d17ecbdb08c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:38.218208 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea34e1b-4a9b-4db0-b018-8d17ecbdb08c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2ea34e1b-4a9b-4db0-b018-8d17ecbdb08c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:38.230830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:38.231944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:38.233043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:38.233043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:38.233043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:38.233043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:38.233043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:38.233043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:38.233043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:38.233043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:38.237120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 378/753] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:38.238081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea34e1b-4a9b-4db0-b018-8d17ecbdb08c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:38.267397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:38.306735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351de52-7cb3-4892-b637-1434b3adeb75 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:38.326161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:38.345551 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea34e1b-4a9b-4db0-b018-8d17ecbdb08c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:38.346561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ae4c5-7a94-4fe7-9a9f-e60fcf11d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:38.347322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae4c5-7a94-4fe7-9a9f-e60fcf11d8fa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:38.347664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae4c5-7a94-4fe7-9a9f-e60fcf11d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:38.348665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ae4c5-7a94-4fe7-9a9f-e60fcf11d8fa None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:38.356741 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ae4c5-7a94-4fe7-9a9f-e60fcf11d8fa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:38.358225 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ae4c5-7a94-4fe7-9a9f-e60fcf11d8fa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-313ae4c5-7a94-4fe7-9a9f-e60fcf11d8fa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:38.370534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:38.371761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 376/754] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jul 26 07:12:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:38.372703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ae4c5-7a94-4fe7-9a9f-e60fcf11d8fa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:38.388951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b98dba-fe7b-4ba1-8188-94bb72d5a26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:38.389592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b98dba-fe7b-4ba1-8188-94bb72d5a26d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:38.389911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b98dba-fe7b-4ba1-8188-94bb72d5a26d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:38.390186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b98dba-fe7b-4ba1-8188-94bb72d5a26d None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:38.390989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b98dba-fe7b-4ba1-8188-94bb72d5a26d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:38.391297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b98dba-fe7b-4ba1-8188-94bb72d5a26d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:38.392740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b98dba-fe7b-4ba1-8188-94bb72d5a26d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:38.411784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 379/755] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jul 26 07:12:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:38.414635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b98dba-fe7b-4ba1-8188-94bb72d5a26d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:38.485482 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ae4c5-7a94-4fe7-9a9f-e60fcf11d8fa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:38.520500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61da83-46fe-4aff-9b12-efca812f24ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:38.520760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:38.521135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61da83-46fe-4aff-9b12-efca812f24ba None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:38.521802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61da83-46fe-4aff-9b12-efca812f24ba None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:38.522084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61da83-46fe-4aff-9b12-efca812f24ba None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:38.522867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61da83-46fe-4aff-9b12-efca812f24ba None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:38.523216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61da83-46fe-4aff-9b12-efca812f24ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:38.524770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c61da83-46fe-4aff-9b12-efca812f24ba None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:38.533887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 377/756] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jul 26 07:12:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:38.533887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61da83-46fe-4aff-9b12-efca812f24ba None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:38.543281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b98dba-fe7b-4ba1-8188-94bb72d5a26d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:38.577369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15db5b0-0af9-46d7-a4d6-8561d3c0e91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:38.578034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15db5b0-0af9-46d7-a4d6-8561d3c0e91e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:38.578317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15db5b0-0af9-46d7-a4d6-8561d3c0e91e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:38.578585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15db5b0-0af9-46d7-a4d6-8561d3c0e91e None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:38.578832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:38.579334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15db5b0-0af9-46d7-a4d6-8561d3c0e91e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:38.579797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15db5b0-0af9-46d7-a4d6-8561d3c0e91e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:38.581133 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15db5b0-0af9-46d7-a4d6-8561d3c0e91e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:38.592836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 380/757] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:38.593666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15db5b0-0af9-46d7-a4d6-8561d3c0e91e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:38.657972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61da83-46fe-4aff-9b12-efca812f24ba None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:38.702089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc030e77-8c8c-4656-8e73-995b7b98c4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:38.702652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc030e77-8c8c-4656-8e73-995b7b98c4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:38.702900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc030e77-8c8c-4656-8e73-995b7b98c4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:38.703173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc030e77-8c8c-4656-8e73-995b7b98c4c1 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:38.703978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc030e77-8c8c-4656-8e73-995b7b98c4c1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:38.704300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc030e77-8c8c-4656-8e73-995b7b98c4c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:38.705768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc030e77-8c8c-4656-8e73-995b7b98c4c1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:38.722298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15db5b0-0af9-46d7-a4d6-8561d3c0e91e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:38.725474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 378/758] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jul 26 07:12:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:38.726580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc030e77-8c8c-4656-8e73-995b7b98c4c1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:38.765060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e11a15-b9e0-4191-b474-224b185e48cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:38.765701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e11a15-b9e0-4191-b474-224b185e48cb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:38.766024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e11a15-b9e0-4191-b474-224b185e48cb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:38.766307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e11a15-b9e0-4191-b474-224b185e48cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:38.783899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:38.843898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:38.848974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc030e77-8c8c-4656-8e73-995b7b98c4c1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:38.853066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e11a15-b9e0-4191-b474-224b185e48cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:38.854630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e11a15-b9e0-4191-b474-224b185e48cb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:38.904368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0413253-5b57-4430-9ce6-1b6b122abb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:38.905477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0413253-5b57-4430-9ce6-1b6b122abb0f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:38.906052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0413253-5b57-4430-9ce6-1b6b122abb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:38.906550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0413253-5b57-4430-9ce6-1b6b122abb0f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:38.907609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0413253-5b57-4430-9ce6-1b6b122abb0f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:38.908229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0413253-5b57-4430-9ce6-1b6b122abb0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:38.910014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0413253-5b57-4430-9ce6-1b6b122abb0f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:38.921141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 379/759] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jul 26 07:12:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:38.922553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0413253-5b57-4430-9ce6-1b6b122abb0f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:39.074902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0413253-5b57-4430-9ce6-1b6b122abb0f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:39.134496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 381/760] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:38 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:12:39.134496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e11a15-b9e0-4191-b474-224b185e48cb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:39.137416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76edb4-4045-42f4-8859-8dd487a1f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:39.138092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76edb4-4045-42f4-8859-8dd487a1f733 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:39.138370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76edb4-4045-42f4-8859-8dd487a1f733 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:39.138638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76edb4-4045-42f4-8859-8dd487a1f733 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:39.139474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76edb4-4045-42f4-8859-8dd487a1f733 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=64257d34b2df4af1a2016d56be8d44a8) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:39.139811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76edb4-4045-42f4-8859-8dd487a1f733 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:39.141288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb76edb4-4045-42f4-8859-8dd487a1f733 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:39.157969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 380/761] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:38 2022] GET /identity/v3/role_assignments?user.id=64257d34b2df4af1a2016d56be8d44a8&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:39.157969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76edb4-4045-42f4-8859-8dd487a1f733 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:39.195935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:39.199158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:39.276431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e11a15-b9e0-4191-b474-224b185e48cb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:39.315103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76edb4-4045-42f4-8859-8dd487a1f733 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:39.320215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6113e6-24f9-48a5-98e9-11c02810b0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:39.321248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6113e6-24f9-48a5-98e9-11c02810b0ad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:39.321712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6113e6-24f9-48a5-98e9-11c02810b0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:39.322209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6113e6-24f9-48a5-98e9-11c02810b0ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:39.336308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6113e6-24f9-48a5-98e9-11c02810b0ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:39.338120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6113e6-24f9-48a5-98e9-11c02810b0ad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5a6113e6-24f9-48a5-98e9-11c02810b0ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:39.363981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:39.364950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:39.366000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:39.366000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:39.366000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:39.366000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:39.366381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 382/762] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:39.371578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6113e6-24f9-48a5-98e9-11c02810b0ad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:39.371770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b2176-5248-4e71-810a-0f0a6268541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:39.371770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b2176-5248-4e71-810a-0f0a6268541c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:39.371770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b2176-5248-4e71-810a-0f0a6268541c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:39.372920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3b2176-5248-4e71-810a-0f0a6268541c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:39.373963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b2176-5248-4e71-810a-0f0a6268541c None admin] RBAC: Adding query filter params (interface=public, service_id=043c350fd78c42dcbda86d777737ed6a, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:39.374298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b2176-5248-4e71-810a-0f0a6268541c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:39.376086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3b2176-5248-4e71-810a-0f0a6268541c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:39.386406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 381/763] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jul 26 07:12:39 2022] GET /identity/v3/endpoints?service_id=043c350fd78c42dcbda86d777737ed6a&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:39.390442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b2176-5248-4e71-810a-0f0a6268541c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:39.484993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:39.492081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6113e6-24f9-48a5-98e9-11c02810b0ad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:39.508946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:39.522624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3b2176-5248-4e71-810a-0f0a6268541c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:39.533602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c7ab3-ec64-4cde-95b6-fc6b4d1bc63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:39.535260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c7ab3-ec64-4cde-95b6-fc6b4d1bc63a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:39.535747 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c7ab3-ec64-4cde-95b6-fc6b4d1bc63a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:39.536189 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285c7ab3-ec64-4cde-95b6-fc6b4d1bc63a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:39.551041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c7ab3-ec64-4cde-95b6-fc6b4d1bc63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:39.553429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285c7ab3-ec64-4cde-95b6-fc6b4d1bc63a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:39.559694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334168c-8d48-4a33-af2b-1d8edcdb3d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:39.560437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334168c-8d48-4a33-af2b-1d8edcdb3d0b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:39.560790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334168c-8d48-4a33-af2b-1d8edcdb3d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:39.561164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0334168c-8d48-4a33-af2b-1d8edcdb3d0b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:39.561966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334168c-8d48-4a33-af2b-1d8edcdb3d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:39.562338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334168c-8d48-4a33-af2b-1d8edcdb3d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:39.563956 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0334168c-8d48-4a33-af2b-1d8edcdb3d0b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-285c7ab3-ec64-4cde-95b6-fc6b4d1bc63a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:39.572850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:39.573776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:39.574694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:39.574694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:39.574694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:39.574694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:39.580637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 383/764] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:39.580637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c7ab3-ec64-4cde-95b6-fc6b4d1bc63a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:39.584302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 382/765] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:39.585863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334168c-8d48-4a33-af2b-1d8edcdb3d0b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:39.709453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285c7ab3-ec64-4cde-95b6-fc6b4d1bc63a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:39.718326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0334168c-8d48-4a33-af2b-1d8edcdb3d0b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:39.748275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ed63b-47bb-4be7-8d4c-ec60b3c04a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:39.748906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ed63b-47bb-4be7-8d4c-ec60b3c04a3a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:39.749230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ed63b-47bb-4be7-8d4c-ec60b3c04a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:39.749708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ed63b-47bb-4be7-8d4c-ec60b3c04a3a None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:39.750183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ed63b-47bb-4be7-8d4c-ec60b3c04a3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:39.751550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ed63b-47bb-4be7-8d4c-ec60b3c04a3a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:39.757469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58007913-05b1-4123-8e1c-43440422495e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:39.758131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58007913-05b1-4123-8e1c-43440422495e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:39.758419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58007913-05b1-4123-8e1c-43440422495e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:39.758697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58007913-05b1-4123-8e1c-43440422495e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:39.759404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58007913-05b1-4123-8e1c-43440422495e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:39.759712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58007913-05b1-4123-8e1c-43440422495e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:39.761135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58007913-05b1-4123-8e1c-43440422495e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:39.761923 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 384/766] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jul 26 07:12:39 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:39.763081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ed63b-47bb-4be7-8d4c-ec60b3c04a3a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:39.774606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 383/767] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:39.775971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33f996-0c7b-4f62-80e2-3fb1fbf23fd1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:39.776262 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33f996-0c7b-4f62-80e2-3fb1fbf23fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:39.776540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33f996-0c7b-4f62-80e2-3fb1fbf23fd1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:39.782170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 384/768] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:39.783163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33f996-0c7b-4f62-80e2-3fb1fbf23fd1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:39.783456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:39.801597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:39.920799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ed63b-47bb-4be7-8d4c-ec60b3c04a3a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:39.926596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33f996-0c7b-4f62-80e2-3fb1fbf23fd1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:39.974487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c702f9-6a8f-46bc-9c6e-f62b8ea8917e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:39.975954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c702f9-6a8f-46bc-9c6e-f62b8ea8917e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:39.977443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c702f9-6a8f-46bc-9c6e-f62b8ea8917e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:39.978296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c702f9-6a8f-46bc-9c6e-f62b8ea8917e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:39.984138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791aeac-1ecd-424e-908f-04a2b201a482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:39.985441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791aeac-1ecd-424e-908f-04a2b201a482 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:39.986023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791aeac-1ecd-424e-908f-04a2b201a482 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:39.986599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791aeac-1ecd-424e-908f-04a2b201a482 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:39.990031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c702f9-6a8f-46bc-9c6e-f62b8ea8917e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:39.992098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c702f9-6a8f-46bc-9c6e-f62b8ea8917e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:39.998463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791aeac-1ecd-424e-908f-04a2b201a482 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:40.000481 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791aeac-1ecd-424e-908f-04a2b201a482 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-59c702f9-6a8f-46bc-9c6e-f62b8ea8917e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:40.007719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:40.008983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:40.010266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:40.010266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:40.010266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:40.010266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:40.010266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:40.010266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:40.010266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:40.010266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:40.012862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 385/769] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:40.014537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5884e3-2797-4884-adc0-f4f8b5b5b757 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:40.015013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5884e3-2797-4884-adc0-f4f8b5b5b757 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:40.015486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5884e3-2797-4884-adc0-f4f8b5b5b757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0791aeac-1ecd-424e-908f-04a2b201a482 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:40.021271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:40.022232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:40.023232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:40.023232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:40.023232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:40.023232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:40.023232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:40.023232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:40.023232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:40.023232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:40.023232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 385/770] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:40.024091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791aeac-1ecd-424e-908f-04a2b201a482 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:40.047917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5b5884e3-2797-4884-adc0-f4f8b5b5b757 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:40.049485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5b5884e3-2797-4884-adc0-f4f8b5b5b757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:40.122648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:40.126664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 386/771] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:40.127963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5884e3-2797-4884-adc0-f4f8b5b5b757 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:40.131878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:40.172622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791aeac-1ecd-424e-908f-04a2b201a482 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:40.216468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688a7be-304c-448e-892a-44a8879b08d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:40.217117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688a7be-304c-448e-892a-44a8879b08d7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:40.217398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688a7be-304c-448e-892a-44a8879b08d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:40.217671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688a7be-304c-448e-892a-44a8879b08d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:40.218477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688a7be-304c-448e-892a-44a8879b08d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:40.218782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688a7be-304c-448e-892a-44a8879b08d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:40.220729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688a7be-304c-448e-892a-44a8879b08d7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:40.239708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 386/772] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:40.240818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688a7be-304c-448e-892a-44a8879b08d7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:40.246199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5884e3-2797-4884-adc0-f4f8b5b5b757 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:40.290669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f86d1-0484-4f94-adab-f6054ca1b82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:40.291310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f86d1-0484-4f94-adab-f6054ca1b82b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:40.291594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f86d1-0484-4f94-adab-f6054ca1b82b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:40.291934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f86d1-0484-4f94-adab-f6054ca1b82b None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:40.293096 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f86d1-0484-4f94-adab-f6054ca1b82b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:40.293403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f86d1-0484-4f94-adab-f6054ca1b82b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:40.295606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f86d1-0484-4f94-adab-f6054ca1b82b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:40.317446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 387/773] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:40.317446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f86d1-0484-4f94-adab-f6054ca1b82b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:40.366147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688a7be-304c-448e-892a-44a8879b08d7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:40.398584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:40.412867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:40.421979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7a22f-c495-43a1-9932-27b8e986354c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:40.423091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7a22f-c495-43a1-9932-27b8e986354c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:40.423375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7a22f-c495-43a1-9932-27b8e986354c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:40.423653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea7a22f-c495-43a1-9932-27b8e986354c None admin] PATH_INFO: `/v3/services/network` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:40.424428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7a22f-c495-43a1-9932-27b8e986354c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:40.426246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea7a22f-c495-43a1-9932-27b8e986354c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9ea7a22f-c495-43a1-9932-27b8e986354c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:40.438573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:40.439673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 07:12:40.440557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:40.444924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 387/774] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:40 2022] GET /identity/v3/services/network => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:40.446517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7a22f-c495-43a1-9932-27b8e986354c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:40.449651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f86d1-0484-4f94-adab-f6054ca1b82b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:40.487377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad469cbf-ba48-455b-a71c-0560eea46f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:40.488979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad469cbf-ba48-455b-a71c-0560eea46f56 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:40.489520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad469cbf-ba48-455b-a71c-0560eea46f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:40.490206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad469cbf-ba48-455b-a71c-0560eea46f56 None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/6d50427014064f099af87932c77a849a/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:40.492755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad469cbf-ba48-455b-a71c-0560eea46f56 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=6d50427014064f099af87932c77a849a, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:40.494604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad469cbf-ba48-455b-a71c-0560eea46f56 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:40.513589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 388/775] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:12:40 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/6d50427014064f099af87932c77a849a/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:12:40.515194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad469cbf-ba48-455b-a71c-0560eea46f56 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:40.578266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea7a22f-c495-43a1-9932-27b8e986354c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:40.644446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2f2c5-7603-4cb1-a0a0-951e6716d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:40.645601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f2c5-7603-4cb1-a0a0-951e6716d6df None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:40.646151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f2c5-7603-4cb1-a0a0-951e6716d6df None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:40.646704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db2f2c5-7603-4cb1-a0a0-951e6716d6df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:40.648337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2f2c5-7603-4cb1-a0a0-951e6716d6df None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=77fb7ab7232b48a286b7798beca19f58) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:40.648893 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2f2c5-7603-4cb1-a0a0-951e6716d6df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:40.651699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db2f2c5-7603-4cb1-a0a0-951e6716d6df None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:40.662997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 388/776] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:40 2022] GET /identity/v3/role_assignments?user.id=77fb7ab7232b48a286b7798beca19f58&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:40.668814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad469cbf-ba48-455b-a71c-0560eea46f56 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:40.699824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:40.720160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04077b-9561-4985-93ae-2de82e93af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:40.721247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04077b-9561-4985-93ae-2de82e93af74 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:40.721873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04077b-9561-4985-93ae-2de82e93af74 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:40.722403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04077b-9561-4985-93ae-2de82e93af74 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:40.723685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04077b-9561-4985-93ae-2de82e93af74 None admin] RBAC: Adding query filter params (name=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:40.724239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04077b-9561-4985-93ae-2de82e93af74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:40.726996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04077b-9561-4985-93ae-2de82e93af74 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:40.736990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 389/777] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:12:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:40.747182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2f2c5-7603-4cb1-a0a0-951e6716d6df None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:40.889813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:40.975798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db2f2c5-7603-4cb1-a0a0-951e6716d6df None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:41.042871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5606b50-8d27-47e0-92ee-2393bde8a007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:41.044040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5606b50-8d27-47e0-92ee-2393bde8a007 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:41.044578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5606b50-8d27-47e0-92ee-2393bde8a007 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:41.044986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5606b50-8d27-47e0-92ee-2393bde8a007 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:41.046559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5606b50-8d27-47e0-92ee-2393bde8a007 None admin] RBAC: Adding query filter params (type=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:41.047175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5606b50-8d27-47e0-92ee-2393bde8a007 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:41.049816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5606b50-8d27-47e0-92ee-2393bde8a007 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:41.064879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 389/778] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:12:40 2022] GET /identity/v3/services?type=network => generated 350 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:41.073708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04077b-9561-4985-93ae-2de82e93af74 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:41.280893 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04077b-9561-4985-93ae-2de82e93af74 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:41.360102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05d4b2-72c6-47e0-982d-3d9d48cd5a42 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:41.360294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05d4b2-72c6-47e0-982d-3d9d48cd5a42 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:41.360552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc05d4b2-72c6-47e0-982d-3d9d48cd5a42 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:41.362289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 390/779] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:12:41.370404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c1e16-d7ad-4fd7-a8f0-c3de69d80a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:41.371095 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c1e16-d7ad-4fd7-a8f0-c3de69d80a0d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:41.371377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c1e16-d7ad-4fd7-a8f0-c3de69d80a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:41.371650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c1e16-d7ad-4fd7-a8f0-c3de69d80a0d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:41.372468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c1e16-d7ad-4fd7-a8f0-c3de69d80a0d None admin] RBAC: Adding query filter params (interface=public, service_id=675fd81837af48369906d5eaa0a9b4d5, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:41.372783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c1e16-d7ad-4fd7-a8f0-c3de69d80a0d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:41.374334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1c1e16-d7ad-4fd7-a8f0-c3de69d80a0d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:41.377617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a2098-0830-4fb8-8875-58c132362c82 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:41.378214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a2098-0830-4fb8-8875-58c132362c82 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:41.378214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327a2098-0830-4fb8-8875-58c132362c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:41.405343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 390/780] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jul 26 07:12:41 2022] GET /identity/v3/endpoints?service_id=675fd81837af48369906d5eaa0a9b4d5&interface=public®ion_id=RegionOne => generated 194 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:41.406606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-327a2098-0830-4fb8-8875-58c132362c82 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:41.407895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-327a2098-0830-4fb8-8875-58c132362c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:41.415458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c1e16-d7ad-4fd7-a8f0-c3de69d80a0d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:41.528230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:41.568134 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 391/781] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:12:41.602813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a2098-0830-4fb8-8875-58c132362c82 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:41.617472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1c1e16-d7ad-4fd7-a8f0-c3de69d80a0d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:41.686668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:41.692809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbb89c-c88f-4f40-9808-18395887fa1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:41.693453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbb89c-c88f-4f40-9808-18395887fa1b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:41.693732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbb89c-c88f-4f40-9808-18395887fa1b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:41.694024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acbb89c-c88f-4f40-9808-18395887fa1b None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:41.694689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbb89c-c88f-4f40-9808-18395887fa1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:41.697038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acbb89c-c88f-4f40-9808-18395887fa1b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:41.709642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 391/782] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jul 26 07:12:41 2022] GET /identity/v3/services => generated 772 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:41.753731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327a2098-0830-4fb8-8875-58c132362c82 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:41.792107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935dd3a-d53b-4399-8757-d4291b8ebbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:41.792765 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935dd3a-d53b-4399-8757-d4291b8ebbd2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:41.793045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935dd3a-d53b-4399-8757-d4291b8ebbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:41.793318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935dd3a-d53b-4399-8757-d4291b8ebbd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:41.805269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935dd3a-d53b-4399-8757-d4291b8ebbd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3935dd3a-d53b-4399-8757-d4291b8ebbd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:41.810697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:41.811720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:41.812773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 392/783] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:41.820079 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbb89c-c88f-4f40-9808-18395887fa1b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:41.958068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acbb89c-c88f-4f40-9808-18395887fa1b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:41.981893 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:42.005604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae8aeae-a04f-4370-b5cd-31ce0bfd4039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:42.006379 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8aeae-a04f-4370-b5cd-31ce0bfd4039 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:42.006793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8aeae-a04f-4370-b5cd-31ce0bfd4039 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:42.007122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae8aeae-a04f-4370-b5cd-31ce0bfd4039 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:42.008188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8aeae-a04f-4370-b5cd-31ce0bfd4039 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:42.008596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8aeae-a04f-4370-b5cd-31ce0bfd4039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:42.010336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae8aeae-a04f-4370-b5cd-31ce0bfd4039 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:42.025607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 392/784] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:42.027044 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935dd3a-d53b-4399-8757-d4291b8ebbd2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:42.091436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a71f65-c700-494e-8472-cf89c471f84d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:42.091766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a71f65-c700-494e-8472-cf89c471f84d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:42.091989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a71f65-c700-494e-8472-cf89c471f84d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:42.093808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 393/785] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:12:42.099206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dccffd-2f02-4980-8574-e4da411ab743 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:42.099599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dccffd-2f02-4980-8574-e4da411ab743 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:42.099885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dccffd-2f02-4980-8574-e4da411ab743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:42.120596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-53dccffd-2f02-4980-8574-e4da411ab743 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:42.123250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53dccffd-2f02-4980-8574-e4da411ab743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:42.159815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935dd3a-d53b-4399-8757-d4291b8ebbd2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:42.182236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:42.208813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 394/786] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:12:42.225314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63680022-ded5-4d02-b211-baee951a1ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:42.226008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63680022-ded5-4d02-b211-baee951a1ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:42.226293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63680022-ded5-4d02-b211-baee951a1ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:42.226571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63680022-ded5-4d02-b211-baee951a1ab6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:42.234337 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dccffd-2f02-4980-8574-e4da411ab743 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:42.247709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63680022-ded5-4d02-b211-baee951a1ab6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:42.250643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63680022-ded5-4d02-b211-baee951a1ab6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-63680022-ded5-4d02-b211-baee951a1ab6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:42.275874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:42.276822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:42.277725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:42.277725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:42.277725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:42.277725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:42.283040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 393/787] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:42.288942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63680022-ded5-4d02-b211-baee951a1ab6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:42.420155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dccffd-2f02-4980-8574-e4da411ab743 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:42.440076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:42.476134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26078-30a0-4ffe-86e7-663e08d5c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:42.477147 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26078-30a0-4ffe-86e7-663e08d5c231 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:42.477642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26078-30a0-4ffe-86e7-663e08d5c231 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:42.478133 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc26078-30a0-4ffe-86e7-663e08d5c231 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:42.478975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc26078-30a0-4ffe-86e7-663e08d5c231 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:42.480549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc26078-30a0-4ffe-86e7-663e08d5c231 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:42.484561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63680022-ded5-4d02-b211-baee951a1ab6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-2bc26078-30a0-4ffe-86e7-663e08d5c231 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:42.492822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:42.493850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 26 07:12:42.494703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:42.497623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 395/788] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Jul 26 07:12:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:42.505061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26078-30a0-4ffe-86e7-663e08d5c231 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:42.559611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84afd2-d854-4896-88ef-e81546caa484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:42.560251 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84afd2-d854-4896-88ef-e81546caa484 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:42.560535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84afd2-d854-4896-88ef-e81546caa484 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:42.560813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84afd2-d854-4896-88ef-e81546caa484 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:42.561529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84afd2-d854-4896-88ef-e81546caa484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:42.561873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84afd2-d854-4896-88ef-e81546caa484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:42.563264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b84afd2-d854-4896-88ef-e81546caa484 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:42.593045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 394/789] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:42.596001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84afd2-d854-4896-88ef-e81546caa484 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:42.645647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:42.700438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc26078-30a0-4ffe-86e7-663e08d5c231 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:42.773944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d987a-a832-4b78-9a44-d15bc87db435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:42.775757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d987a-a832-4b78-9a44-d15bc87db435 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:42.776418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d987a-a832-4b78-9a44-d15bc87db435 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:42.778015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d987a-a832-4b78-9a44-d15bc87db435 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:42.779107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d987a-a832-4b78-9a44-d15bc87db435 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:42.780337 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d987a-a832-4b78-9a44-d15bc87db435 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:42.784878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d987a-a832-4b78-9a44-d15bc87db435 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:42.797818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 396/790] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jul 26 07:12:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:42.800331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d987a-a832-4b78-9a44-d15bc87db435 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:42.817488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84afd2-d854-4896-88ef-e81546caa484 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:42.852222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:42.887096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7aee-7e1e-4466-a5cc-716a22133186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:42.887760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7aee-7e1e-4466-a5cc-716a22133186 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:42.888039 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7aee-7e1e-4466-a5cc-716a22133186 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:42.888314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7aee-7e1e-4466-a5cc-716a22133186 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:42.898682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf7aee-7e1e-4466-a5cc-716a22133186 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:42.900139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf7aee-7e1e-4466-a5cc-716a22133186 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-99bf7aee-7e1e-4466-a5cc-716a22133186 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:42.920967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:42.922025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:42.922952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 395/791] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:42.930240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7aee-7e1e-4466-a5cc-716a22133186 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:42.976254 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d987a-a832-4b78-9a44-d15bc87db435 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:43.038981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6a5c5-49ca-47ce-9d65-a019e2348a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:43.039535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6a5c5-49ca-47ce-9d65-a019e2348a6c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.039818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6a5c5-49ca-47ce-9d65-a019e2348a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.040093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6a5c5-49ca-47ce-9d65-a019e2348a6c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.040781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6a5c5-49ca-47ce-9d65-a019e2348a6c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:43.041086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6a5c5-49ca-47ce-9d65-a019e2348a6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:43.042484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6a5c5-49ca-47ce-9d65-a019e2348a6c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:43.058892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 397/792] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Jul 26 07:12:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:43.061084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6a5c5-49ca-47ce-9d65-a019e2348a6c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:43.080630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:43.100998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7aee-7e1e-4466-a5cc-716a22133186 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:43.146801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751be0d-d8aa-459e-b0f3-848164be0979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:43.147445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751be0d-d8aa-459e-b0f3-848164be0979 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.147879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751be0d-d8aa-459e-b0f3-848164be0979 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.148166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751be0d-d8aa-459e-b0f3-848164be0979 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.149384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751be0d-d8aa-459e-b0f3-848164be0979 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:43.149699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751be0d-d8aa-459e-b0f3-848164be0979 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:43.151438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751be0d-d8aa-459e-b0f3-848164be0979 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:43.175255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 396/793] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jul 26 07:12:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:43.176698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf18e1-bd4a-4d9a-84d3-78e60318ec35 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.176991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf18e1-bd4a-4d9a-84d3-78e60318ec35 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.177273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf18e1-bd4a-4d9a-84d3-78e60318ec35 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.181812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 397/794] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:43.185537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf18e1-bd4a-4d9a-84d3-78e60318ec35 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:43.212940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:43.216947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6a5c5-49ca-47ce-9d65-a019e2348a6c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:43.257024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0c0c67-995e-48d5-b910-375511db8d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:43.258169 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c0c67-995e-48d5-b910-375511db8d73 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.258698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c0c67-995e-48d5-b910-375511db8d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.259169 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0c0c67-995e-48d5-b910-375511db8d73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.260158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c0c67-995e-48d5-b910-375511db8d73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:43.261829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0c0c67-995e-48d5-b910-375511db8d73 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:43.288520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 398/795] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:12:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:43.290206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eed4c5-20e0-405d-8475-7829fb73e160 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.290677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eed4c5-20e0-405d-8475-7829fb73e160 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.291129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eed4c5-20e0-405d-8475-7829fb73e160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.312442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c2eed4c5-20e0-405d-8475-7829fb73e160 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:43.314240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2eed4c5-20e0-405d-8475-7829fb73e160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:43.344082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf18e1-bd4a-4d9a-84d3-78e60318ec35 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:43.397812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32271782-5abe-49c6-a092-7094e2c15b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:43.398445 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 399/796] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:43.398555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32271782-5abe-49c6-a092-7094e2c15b81 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.398909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32271782-5abe-49c6-a092-7094e2c15b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.399252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32271782-5abe-49c6-a092-7094e2c15b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.400603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b13ba7-77ab-4a68-8b46-0509c8062a91 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.401092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b13ba7-77ab-4a68-8b46-0509c8062a91 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.405770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b13ba7-77ab-4a68-8b46-0509c8062a91 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.407220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:43.408292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 400/797] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:43.413483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32271782-5abe-49c6-a092-7094e2c15b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:43.414245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a8cd-c038-45d3-9df0-f537fb6396ea None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.414739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a8cd-c038-45d3-9df0-f537fb6396ea None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.415162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32271782-5abe-49c6-a092-7094e2c15b81 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:43.415454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4126a8cd-c038-45d3-9df0-f537fb6396ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-32271782-5abe-49c6-a092-7094e2c15b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:43.432395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:43.433353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:43.434815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:43.434815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:43.434815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:43.434815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:43.434815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 398/798] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:43.440135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32271782-5abe-49c6-a092-7094e2c15b81 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:43.458570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4126a8cd-c038-45d3-9df0-f537fb6396ea None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:43.460078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4126a8cd-c038-45d3-9df0-f537fb6396ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:43.550802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 401/799] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:12:43.552238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126a8cd-c038-45d3-9df0-f537fb6396ea None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:43.573786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32271782-5abe-49c6-a092-7094e2c15b81 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:43.585704 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:43.622126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6648636-ef52-4e51-be88-558daa35cd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:43.622890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6648636-ef52-4e51-be88-558daa35cd92 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.623373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6648636-ef52-4e51-be88-558daa35cd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.623890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6648636-ef52-4e51-be88-558daa35cd92 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.624669 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6648636-ef52-4e51-be88-558daa35cd92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:43.625106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6648636-ef52-4e51-be88-558daa35cd92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:43.627364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6648636-ef52-4e51-be88-558daa35cd92 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:43.644294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 399/800] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:43.645588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6648636-ef52-4e51-be88-558daa35cd92 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:43.693540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4126a8cd-c038-45d3-9df0-f537fb6396ea None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:43.700589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:43.733469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2316ca3-792d-4020-ab60-4317aa44061b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:43.734624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2316ca3-792d-4020-ab60-4317aa44061b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.734972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2316ca3-792d-4020-ab60-4317aa44061b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.735335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2316ca3-792d-4020-ab60-4317aa44061b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.747395 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2316ca3-792d-4020-ab60-4317aa44061b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b2316ca3-792d-4020-ab60-4317aa44061b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:43.753249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:43.754331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:43.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:43.759554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 402/801] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:43.760489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2316ca3-792d-4020-ab60-4317aa44061b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:43.761005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6648636-ef52-4e51-be88-558daa35cd92 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:43.798033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db774c7-a1b8-4726-9637-524bee02d9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:43.799086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db774c7-a1b8-4726-9637-524bee02d9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.799587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db774c7-a1b8-4726-9637-524bee02d9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.800094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db774c7-a1b8-4726-9637-524bee02d9d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.811794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db774c7-a1b8-4726-9637-524bee02d9d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0db774c7-a1b8-4726-9637-524bee02d9d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:43.817180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:43.818248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:43.819083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:43.821988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 400/802] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:43.823072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db774c7-a1b8-4726-9637-524bee02d9d8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:43.849844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:43.882176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2316ca3-792d-4020-ab60-4317aa44061b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:43.934591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196b261-5c60-4cff-ba0a-ed5724053d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:43.935286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196b261-5c60-4cff-ba0a-ed5724053d7d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.935569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196b261-5c60-4cff-ba0a-ed5724053d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.935842 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f196b261-5c60-4cff-ba0a-ed5724053d7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.943239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db774c7-a1b8-4726-9637-524bee02d9d8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:43.945230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196b261-5c60-4cff-ba0a-ed5724053d7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:43.947644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f196b261-5c60-4cff-ba0a-ed5724053d7d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:43.955988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f196b261-5c60-4cff-ba0a-ed5724053d7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:43.964042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:43.964959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:43.965881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:43.965881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:43.965881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:43.965881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:43.965881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:43.965881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:43.965881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:43.965881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:43.965881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 403/803] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:43.966859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196b261-5c60-4cff-ba0a-ed5724053d7d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:43.986388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186442a-5d6e-4bce-90f8-10e7199117f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:43.987348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186442a-5d6e-4bce-90f8-10e7199117f9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:43.987944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186442a-5d6e-4bce-90f8-10e7199117f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:43.988436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186442a-5d6e-4bce-90f8-10e7199117f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:43.989398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186442a-5d6e-4bce-90f8-10e7199117f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:43.989981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186442a-5d6e-4bce-90f8-10e7199117f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:43.991562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186442a-5d6e-4bce-90f8-10e7199117f9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:44.001088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 401/804] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:44.002372 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186442a-5d6e-4bce-90f8-10e7199117f9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:44.104942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f196b261-5c60-4cff-ba0a-ed5724053d7d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:44.111927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186442a-5d6e-4bce-90f8-10e7199117f9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:44.147528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f865d-fbe0-43a8-9042-42a905cc04b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:44.148170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f865d-fbe0-43a8-9042-42a905cc04b7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.148437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f865d-fbe0-43a8-9042-42a905cc04b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.148706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8f865d-fbe0-43a8-9042-42a905cc04b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.149074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9751c17-219b-40fd-a71e-d806cbf13664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:44.149431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f865d-fbe0-43a8-9042-42a905cc04b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:44.149762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9751c17-219b-40fd-a71e-d806cbf13664 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.149857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f865d-fbe0-43a8-9042-42a905cc04b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:44.150043 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9751c17-219b-40fd-a71e-d806cbf13664 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.150327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9751c17-219b-40fd-a71e-d806cbf13664 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.151087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9751c17-219b-40fd-a71e-d806cbf13664 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:44.151177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8f865d-fbe0-43a8-9042-42a905cc04b7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:44.151396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9751c17-219b-40fd-a71e-d806cbf13664 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:44.152822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9751c17-219b-40fd-a71e-d806cbf13664 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:44.153728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:44.166362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 404/805] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:44.166362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f865d-fbe0-43a8-9042-42a905cc04b7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:44.174545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 402/806] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:44.174545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c6c0-3f50-49c6-921b-bd7cfe492e35 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.174545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c6c0-3f50-49c6-921b-bd7cfe492e35 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.174545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c6c0-3f50-49c6-921b-bd7cfe492e35 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.174545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 403/807] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:44.174545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0c6c0-3f50-49c6-921b-bd7cfe492e35 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:44.207734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:44.269399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8f865d-fbe0-43a8-9042-42a905cc04b7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:44.284182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0c6c0-3f50-49c6-921b-bd7cfe492e35 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:44.306227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81910ecb-8211-4d31-8dfa-2457773d1df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:44.306880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81910ecb-8211-4d31-8dfa-2457773d1df4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.307162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81910ecb-8211-4d31-8dfa-2457773d1df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.307434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81910ecb-8211-4d31-8dfa-2457773d1df4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.317821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81910ecb-8211-4d31-8dfa-2457773d1df4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:44.318251 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9259f8-0ea1-4878-8554-94097ad87728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:44.318899 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9259f8-0ea1-4878-8554-94097ad87728 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.319182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9259f8-0ea1-4878-8554-94097ad87728 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.319276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81910ecb-8211-4d31-8dfa-2457773d1df4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:44.319453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9259f8-0ea1-4878-8554-94097ad87728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.331172 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9259f8-0ea1-4878-8554-94097ad87728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:44.332683 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9259f8-0ea1-4878-8554-94097ad87728 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-81910ecb-8211-4d31-8dfa-2457773d1df4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:44.333991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:44.334856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:44.335775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:44.335775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:44.335775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:44.335775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:44.335775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 405/808] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:44.336713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70dcd7-742d-4049-ab87-f35b94f92f0d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.336713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70dcd7-742d-4049-ab87-f35b94f92f0d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.336868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70dcd7-742d-4049-ab87-f35b94f92f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ed9259f8-0ea1-4878-8554-94097ad87728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:44.353495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:44.354494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:44.356048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:44.356048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:44.356048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:44.356048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:44.356048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 404/809] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:44.356048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9259f8-0ea1-4878-8554-94097ad87728 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:44.360722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9b70dcd7-742d-4049-ab87-f35b94f92f0d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:44.361785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9b70dcd7-742d-4049-ab87-f35b94f92f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:44.410281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:44.449998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 406/810] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:44.449998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70dcd7-742d-4049-ab87-f35b94f92f0d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:44.454939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:44.472856 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9259f8-0ea1-4878-8554-94097ad87728 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:44.508581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceef1c-882b-420b-a2bc-77962fb1513b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:44.509271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceef1c-882b-420b-a2bc-77962fb1513b None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.509553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceef1c-882b-420b-a2bc-77962fb1513b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.509882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceef1c-882b-420b-a2bc-77962fb1513b None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/64257d34b2df4af1a2016d56be8d44a8/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.511876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceef1c-882b-420b-a2bc-77962fb1513b None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=64257d34b2df4af1a2016d56be8d44a8, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:44.513327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aceef1c-882b-420b-a2bc-77962fb1513b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:44.527728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 405/811] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:12:44 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/64257d34b2df4af1a2016d56be8d44a8/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:12:44.528620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceef1c-882b-420b-a2bc-77962fb1513b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:44.582997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70dcd7-742d-4049-ab87-f35b94f92f0d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:44.623770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504b666-3cae-4e16-a49c-be9e86860ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:44.624467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504b666-3cae-4e16-a49c-be9e86860ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.624752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504b666-3cae-4e16-a49c-be9e86860ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.625072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4504b666-3cae-4e16-a49c-be9e86860ae0 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.625910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504b666-3cae-4e16-a49c-be9e86860ae0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:44.626182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504b666-3cae-4e16-a49c-be9e86860ae0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:44.627983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4504b666-3cae-4e16-a49c-be9e86860ae0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:44.641532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 407/812] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:44.642567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504b666-3cae-4e16-a49c-be9e86860ae0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:44.675007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceef1c-882b-420b-a2bc-77962fb1513b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:44.696942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:44.715142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83873f8-c4a7-4ab9-b06f-1dcd1a25a1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:44.715788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83873f8-c4a7-4ab9-b06f-1dcd1a25a1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.716072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83873f8-c4a7-4ab9-b06f-1dcd1a25a1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.716349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83873f8-c4a7-4ab9-b06f-1dcd1a25a1d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.717057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83873f8-c4a7-4ab9-b06f-1dcd1a25a1d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:44.717362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83873f8-c4a7-4ab9-b06f-1dcd1a25a1d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:44.718854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83873f8-c4a7-4ab9-b06f-1dcd1a25a1d8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:44.732457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 406/813] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:44.732994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83873f8-c4a7-4ab9-b06f-1dcd1a25a1d8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:44.738768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:44.785811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4504b666-3cae-4e16-a49c-be9e86860ae0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:44.835065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a8b0b-4588-4c9b-a090-fe349ae12acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:44.836201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a8b0b-4588-4c9b-a090-fe349ae12acc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.836957 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a8b0b-4588-4c9b-a090-fe349ae12acc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.837625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a8b0b-4588-4c9b-a090-fe349ae12acc None admin] PATH_INFO: `/v3/services/network` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.838489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a8b0b-4588-4c9b-a090-fe349ae12acc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:44.840812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1a8b0b-4588-4c9b-a090-fe349ae12acc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:44.842631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83873f8-c4a7-4ab9-b06f-1dcd1a25a1d8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ce1a8b0b-4588-4c9b-a090-fe349ae12acc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:44.859328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:44.860440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 26 07:12:44.861281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:44.864275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 408/814] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:44 2022] GET /identity/v3/services/network => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:44.865273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a8b0b-4588-4c9b-a090-fe349ae12acc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:44.881072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b39bf-c69e-49d4-bd3e-a9b3efd4c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:44.881483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b39bf-c69e-49d4-bd3e-a9b3efd4c524 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:44.881894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b39bf-c69e-49d4-bd3e-a9b3efd4c524 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:44.882263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b39bf-c69e-49d4-bd3e-a9b3efd4c524 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:44.890014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b39bf-c69e-49d4-bd3e-a9b3efd4c524 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:44.891632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b39bf-c69e-49d4-bd3e-a9b3efd4c524 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-946b39bf-c69e-49d4-bd3e-a9b3efd4c524 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:44.905602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:44.906822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:44.907990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:44.907990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:44.907990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:44.907990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:44.907990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:44.907990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:44.907990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:44.907990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:44.907990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:12:44.907990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:44.909866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 407/815] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jul 26 07:12:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:44.910874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b39bf-c69e-49d4-bd3e-a9b3efd4c524 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:44.978161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:45.000483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a8b0b-4588-4c9b-a090-fe349ae12acc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:45.004010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:45.015159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b39bf-c69e-49d4-bd3e-a9b3efd4c524 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:45.045383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05607af5-e1de-4d9f-b19f-6466d96d3bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:45.046372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05607af5-e1de-4d9f-b19f-6466d96d3bcd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.046794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05607af5-e1de-4d9f-b19f-6466d96d3bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.047234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05607af5-e1de-4d9f-b19f-6466d96d3bcd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.056336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05607af5-e1de-4d9f-b19f-6466d96d3bcd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:45.057457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342fd70-7760-48ed-a475-ee8a56848a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:45.058124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342fd70-7760-48ed-a475-ee8a56848a88 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.058406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342fd70-7760-48ed-a475-ee8a56848a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.058666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05607af5-e1de-4d9f-b19f-6466d96d3bcd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:45.058786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342fd70-7760-48ed-a475-ee8a56848a88 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.059364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342fd70-7760-48ed-a475-ee8a56848a88 None admin] RBAC: Adding query filter params (name=network) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:45.059663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342fd70-7760-48ed-a475-ee8a56848a88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:45.061300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342fd70-7760-48ed-a475-ee8a56848a88 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:45.071926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 408/816] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:12:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-05607af5-e1de-4d9f-b19f-6466d96d3bcd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:45.072426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:45.073364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:45.074987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:45.074987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:45.074987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:45.074987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:45.074987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:45.074987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:45.074987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:45.074987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:45.074987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:12:45.074987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:45.075527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342fd70-7760-48ed-a475-ee8a56848a88 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:45.075607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 409/817] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jul 26 07:12:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:45.077464 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05607af5-e1de-4d9f-b19f-6466d96d3bcd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:45.186309 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05607af5-e1de-4d9f-b19f-6466d96d3bcd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:45.213463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342fd70-7760-48ed-a475-ee8a56848a88 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:45.238441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c72558-e228-4530-bbd4-ea6008fc442f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:45.239425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c72558-e228-4530-bbd4-ea6008fc442f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.239830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c72558-e228-4530-bbd4-ea6008fc442f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.240242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c72558-e228-4530-bbd4-ea6008fc442f None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.240939 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c72558-e228-4530-bbd4-ea6008fc442f None admin] RBAC: Adding query filter params (type=network) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:45.241402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c72558-e228-4530-bbd4-ea6008fc442f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:45.241986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:45.243141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c72558-e228-4530-bbd4-ea6008fc442f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:45.251685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324874dd-9a4f-4233-9812-1d150950b9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:45.252627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324874dd-9a4f-4233-9812-1d150950b9ea None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.253082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324874dd-9a4f-4233-9812-1d150950b9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.253559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324874dd-9a4f-4233-9812-1d150950b9ea None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.254707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324874dd-9a4f-4233-9812-1d150950b9ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:45.254822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 410/818] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:12:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:45.255289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324874dd-9a4f-4233-9812-1d150950b9ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:45.255800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c72558-e228-4530-bbd4-ea6008fc442f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:45.257309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324874dd-9a4f-4233-9812-1d150950b9ea None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:45.276616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 409/819] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jul 26 07:12:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:45.277863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324874dd-9a4f-4233-9812-1d150950b9ea None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:45.296837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:45.384189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324874dd-9a4f-4233-9812-1d150950b9ea None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:45.390614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c72558-e228-4530-bbd4-ea6008fc442f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:45.423396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4f763-526b-42ae-8714-576699d0d1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:45.424367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4f763-526b-42ae-8714-576699d0d1db None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.424828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4f763-526b-42ae-8714-576699d0d1db None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.425489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4f763-526b-42ae-8714-576699d0d1db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.427715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d778f-9f67-43a2-92b3-f0c004fc808c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:45.428345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d778f-9f67-43a2-92b3-f0c004fc808c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.428568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4f763-526b-42ae-8714-576699d0d1db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:45.428671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d778f-9f67-43a2-92b3-f0c004fc808c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.428883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d778f-9f67-43a2-92b3-f0c004fc808c None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.429688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d778f-9f67-43a2-92b3-f0c004fc808c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:45.430003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4f763-526b-42ae-8714-576699d0d1db None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:45.430100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d778f-9f67-43a2-92b3-f0c004fc808c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:45.431429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d778f-9f67-43a2-92b3-f0c004fc808c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:45.450786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 411/820] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jul 26 07:12:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:45.451635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d778f-9f67-43a2-92b3-f0c004fc808c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:45.454547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 410/821] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:12:45 2022] POST /identity/v3/endpoints => generated 327 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:45.457021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51b7d7-9f4b-4cbe-bf2e-2f133e6eebfb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.457502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51b7d7-9f4b-4cbe-bf2e-2f133e6eebfb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.458021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51b7d7-9f4b-4cbe-bf2e-2f133e6eebfb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.459852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 411/822] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:45.460798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51b7d7-9f4b-4cbe-bf2e-2f133e6eebfb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:45.542174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:45.567566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:45.586882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d778f-9f67-43a2-92b3-f0c004fc808c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:45.625859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51b7d7-9f4b-4cbe-bf2e-2f133e6eebfb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:45.634614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e85948d-8472-4fbd-8ffc-dc42235db883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:45.634910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85948d-8472-4fbd-8ffc-dc42235db883 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.635178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85948d-8472-4fbd-8ffc-dc42235db883 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.635455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e85948d-8472-4fbd-8ffc-dc42235db883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.648634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85948d-8472-4fbd-8ffc-dc42235db883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:45.650462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e85948d-8472-4fbd-8ffc-dc42235db883 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8e85948d-8472-4fbd-8ffc-dc42235db883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:45.668500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:45.669443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:45.670432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:45.670432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:45.670432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:45.670432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:45.673942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 412/823] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:45.675875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27dec1-6965-4c2b-ac7e-40a82fc24030 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.676010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100c52a-a7f4-4a27-a2d8-ba836232dc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:45.676656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100c52a-a7f4-4a27-a2d8-ba836232dc98 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.676944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100c52a-a7f4-4a27-a2d8-ba836232dc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.677218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0100c52a-a7f4-4a27-a2d8-ba836232dc98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.677517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27dec1-6965-4c2b-ac7e-40a82fc24030 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.678038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27dec1-6965-4c2b-ac7e-40a82fc24030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.702011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7a27dec1-6965-4c2b-ac7e-40a82fc24030 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:45.703293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7a27dec1-6965-4c2b-ac7e-40a82fc24030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:45.765540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100c52a-a7f4-4a27-a2d8-ba836232dc98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:45.767024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0100c52a-a7f4-4a27-a2d8-ba836232dc98 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:45.785922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 413/824] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:45.786722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27dec1-6965-4c2b-ac7e-40a82fc24030 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:45.812530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:45.871493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:45.884812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 412/825] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:12:45.885995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100c52a-a7f4-4a27-a2d8-ba836232dc98 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:45.910223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27dec1-6965-4c2b-ac7e-40a82fc24030 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:45.949327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b704f9-6839-4df7-8c3a-1091f954958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:45.950286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b704f9-6839-4df7-8c3a-1091f954958a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:45.950570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b704f9-6839-4df7-8c3a-1091f954958a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:45.950838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b704f9-6839-4df7-8c3a-1091f954958a None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:45.951552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b704f9-6839-4df7-8c3a-1091f954958a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:45.951856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b704f9-6839-4df7-8c3a-1091f954958a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:45.953318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b704f9-6839-4df7-8c3a-1091f954958a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:45.965924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 414/826] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:45.966836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b704f9-6839-4df7-8c3a-1091f954958a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:46.003600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0100c52a-a7f4-4a27-a2d8-ba836232dc98 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:46.042222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dbd0e-3b06-4240-8ae0-6f8594eb8baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:46.042901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dbd0e-3b06-4240-8ae0-6f8594eb8baa None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.043182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dbd0e-3b06-4240-8ae0-6f8594eb8baa None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.043465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dbd0e-3b06-4240-8ae0-6f8594eb8baa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.044114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dbd0e-3b06-4240-8ae0-6f8594eb8baa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:46.045529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dbd0e-3b06-4240-8ae0-6f8594eb8baa None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-355dbd0e-3b06-4240-8ae0-6f8594eb8baa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:46.055786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:46.057046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 07:12:46.058094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:46.062075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 413/827] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:46.063032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dbd0e-3b06-4240-8ae0-6f8594eb8baa None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:46.066088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:46.091196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b704f9-6839-4df7-8c3a-1091f954958a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:46.122091 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:46.133059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d7dd0-a7a1-4fe1-ac17-36bbfad51081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:46.133860 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d7dd0-a7a1-4fe1-ac17-36bbfad51081 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.134209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d7dd0-a7a1-4fe1-ac17-36bbfad51081 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.134659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d7dd0-a7a1-4fe1-ac17-36bbfad51081 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.147192 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d7dd0-a7a1-4fe1-ac17-36bbfad51081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:46.148913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d7dd0-a7a1-4fe1-ac17-36bbfad51081 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c45d7dd0-a7a1-4fe1-ac17-36bbfad51081 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:46.165597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:46.166778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:46.172680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:46.172680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:46.172680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:46.172680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:46.172680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 415/828] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:46.172680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d7dd0-a7a1-4fe1-ac17-36bbfad51081 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:46.179292 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dbd0e-3b06-4240-8ae0-6f8594eb8baa None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:46.222412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe6ea1-d639-4548-b017-8fc7f15a12a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:46.222987 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe6ea1-d639-4548-b017-8fc7f15a12a5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.223271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe6ea1-d639-4548-b017-8fc7f15a12a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.223548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe6ea1-d639-4548-b017-8fc7f15a12a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.230755 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe6ea1-d639-4548-b017-8fc7f15a12a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:46.232380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe6ea1-d639-4548-b017-8fc7f15a12a5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-04fe6ea1-d639-4548-b017-8fc7f15a12a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:46.244677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:46.245860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:46.247042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:46.247042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:46.247042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:46.247042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:46.247042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:46.247042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:46.247042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:46.247042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:46.247042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 414/829] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:46.247526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe6ea1-d639-4548-b017-8fc7f15a12a5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:46.291058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d7dd0-a7a1-4fe1-ac17-36bbfad51081 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:46.315184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:46.333922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e716e-8307-4194-814f-901f594a2f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:46.334564 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e716e-8307-4194-814f-901f594a2f26 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.334844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e716e-8307-4194-814f-901f594a2f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.335119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e716e-8307-4194-814f-901f594a2f26 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.335819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e716e-8307-4194-814f-901f594a2f26 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:46.336123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e716e-8307-4194-814f-901f594a2f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:46.337515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54e716e-8307-4194-814f-901f594a2f26 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:46.348146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 416/830] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:46.348146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e716e-8307-4194-814f-901f594a2f26 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:46.367187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe6ea1-d639-4548-b017-8fc7f15a12a5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:46.400284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:46.410103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ebcb1-47a8-4124-863b-fa8941514d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:46.410829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ebcb1-47a8-4124-863b-fa8941514d0b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.411118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ebcb1-47a8-4124-863b-fa8941514d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.411402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6ebcb1-47a8-4124-863b-fa8941514d0b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.412108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ebcb1-47a8-4124-863b-fa8941514d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:46.412415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ebcb1-47a8-4124-863b-fa8941514d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:46.413836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6ebcb1-47a8-4124-863b-fa8941514d0b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:46.429024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 415/831] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:46.429482 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ebcb1-47a8-4124-863b-fa8941514d0b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:46.472702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54e716e-8307-4194-814f-901f594a2f26 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:46.525614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92f82a-375b-4e7a-a3e7-905903f957ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:46.526567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92f82a-375b-4e7a-a3e7-905903f957ac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.527019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92f82a-375b-4e7a-a3e7-905903f957ac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.527292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92f82a-375b-4e7a-a3e7-905903f957ac None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.528079 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92f82a-375b-4e7a-a3e7-905903f957ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:46.528389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92f82a-375b-4e7a-a3e7-905903f957ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:46.529877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92f82a-375b-4e7a-a3e7-905903f957ac None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:46.545335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 417/832] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:46.551161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92f82a-375b-4e7a-a3e7-905903f957ac None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:46.576605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6ebcb1-47a8-4124-863b-fa8941514d0b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:46.609480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:46.625629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016cf40-35f9-47e2-adef-66314f323a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:46.626311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016cf40-35f9-47e2-adef-66314f323a52 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.626592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016cf40-35f9-47e2-adef-66314f323a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.626868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6016cf40-35f9-47e2-adef-66314f323a52 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.627748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016cf40-35f9-47e2-adef-66314f323a52 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:46.628156 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016cf40-35f9-47e2-adef-66314f323a52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:46.629558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6016cf40-35f9-47e2-adef-66314f323a52 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:46.642867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 416/833] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:46 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:46.642867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016cf40-35f9-47e2-adef-66314f323a52 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:46.687290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92f82a-375b-4e7a-a3e7-905903f957ac None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:46.694530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:46.735187 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9a622-b012-4c0b-be2e-33afa497b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:46.736227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a622-b012-4c0b-be2e-33afa497b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.736710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a622-b012-4c0b-be2e-33afa497b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.737178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9a622-b012-4c0b-be2e-33afa497b6f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.746643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a622-b012-4c0b-be2e-33afa497b6f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:46.748259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9a622-b012-4c0b-be2e-33afa497b6f1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-16a9a622-b012-4c0b-be2e-33afa497b6f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:46.762841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:46.764047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:46.765634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:46.765634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:46.765634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:46.765634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:46.765634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:46.765634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:46.765634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:46.765634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:46.765634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 418/834] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:46.765634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d587efa-a967-4411-8e59-ce416467eb82 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.766575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d587efa-a967-4411-8e59-ce416467eb82 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.766575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d587efa-a967-4411-8e59-ce416467eb82 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.767595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 419/835] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:46.773233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d587efa-a967-4411-8e59-ce416467eb82 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:46.779444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6016cf40-35f9-47e2-adef-66314f323a52 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:46.825175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533d75e-b142-422b-aaf4-025a7a6eadd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:46.825894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533d75e-b142-422b-aaf4-025a7a6eadd1 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.826172 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533d75e-b142-422b-aaf4-025a7a6eadd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.826447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533d75e-b142-422b-aaf4-025a7a6eadd1 None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/77fb7ab7232b48a286b7798beca19f58/roles/e998fc875b3b45fc8bcab8bff2f1dc5a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.845084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533d75e-b142-422b-aaf4-025a7a6eadd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=77fb7ab7232b48a286b7798beca19f58, role_id=e998fc875b3b45fc8bcab8bff2f1dc5a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:46.846567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533d75e-b142-422b-aaf4-025a7a6eadd1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:46.862655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 417/836] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:12:46 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/77fb7ab7232b48a286b7798beca19f58/roles/e998fc875b3b45fc8bcab8bff2f1dc5a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:12:46.864219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934d7d5-c4fe-432b-b9bc-a1ec1983fdbb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.864507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934d7d5-c4fe-432b-b9bc-a1ec1983fdbb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.864779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934d7d5-c4fe-432b-b9bc-a1ec1983fdbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:46.884972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2934d7d5-c4fe-432b-b9bc-a1ec1983fdbb None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:46.886258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2934d7d5-c4fe-432b-b9bc-a1ec1983fdbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:46.923442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:46.951018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d587efa-a967-4411-8e59-ce416467eb82 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:46.978270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 418/837] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:46.997460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ab53e-7d46-4792-8b04-36a734bb20bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:46.997673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934d7d5-c4fe-432b-b9bc-a1ec1983fdbb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:46.998580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ab53e-7d46-4792-8b04-36a734bb20bf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:46.999036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ab53e-7d46-4792-8b04-36a734bb20bf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:46.999478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ab53e-7d46-4792-8b04-36a734bb20bf None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:47.000712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ab53e-7d46-4792-8b04-36a734bb20bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:47.001229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ab53e-7d46-4792-8b04-36a734bb20bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:47.004067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5ab53e-7d46-4792-8b04-36a734bb20bf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:47.021049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 420/838] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:47.028834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ab53e-7d46-4792-8b04-36a734bb20bf None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:47.065454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:47.140451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934d7d5-c4fe-432b-b9bc-a1ec1983fdbb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:47.168426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ab53e-7d46-4792-8b04-36a734bb20bf None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:47.179187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb051e2-468b-4758-b319-360a4b832f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:47.180067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb051e2-468b-4758-b319-360a4b832f43 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:47.180424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb051e2-468b-4758-b319-360a4b832f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:47.180772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb051e2-468b-4758-b319-360a4b832f43 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:47.192226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb051e2-468b-4758-b319-360a4b832f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-feb051e2-468b-4758-b319-360a4b832f43 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:47.197972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:47.199787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:12:47.201373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:47.203578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 419/839] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:47.207953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb051e2-468b-4758-b319-360a4b832f43 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:47.211526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1e66-4786-4ad6-96a8-3ff015149a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:47.213035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1e66-4786-4ad6-96a8-3ff015149a13 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:47.213035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1e66-4786-4ad6-96a8-3ff015149a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:47.213035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1e66-4786-4ad6-96a8-3ff015149a13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:47.214001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb1e66-4786-4ad6-96a8-3ff015149a13 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=6d50427014064f099af87932c77a849a) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:47.214402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb1e66-4786-4ad6-96a8-3ff015149a13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:47.215938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb1e66-4786-4ad6-96a8-3ff015149a13 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:47.223202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 421/840] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:47 2022] GET /identity/v3/role_assignments?user.id=6d50427014064f099af87932c77a849a&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:47.243954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:47.331261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb051e2-468b-4758-b319-360a4b832f43 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:47.373023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264cf20-b8af-4afa-9ca7-3fb339f614a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:47.373664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264cf20-b8af-4afa-9ca7-3fb339f614a3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:47.373981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264cf20-b8af-4afa-9ca7-3fb339f614a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:47.374256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2264cf20-b8af-4afa-9ca7-3fb339f614a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:47.374997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264cf20-b8af-4afa-9ca7-3fb339f614a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:47.375303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264cf20-b8af-4afa-9ca7-3fb339f614a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:47.376823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2264cf20-b8af-4afa-9ca7-3fb339f614a3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:47.387172 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 420/841] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:12:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:47.392740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1e66-4786-4ad6-96a8-3ff015149a13 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:47.514158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:47.528506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1e66-4786-4ad6-96a8-3ff015149a13 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:47.575971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cabb2-f285-4da1-bf0b-ebd18e4f6ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:47.577057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cabb2-f285-4da1-bf0b-ebd18e4f6ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:47.577506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cabb2-f285-4da1-bf0b-ebd18e4f6ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:47.577980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cabb2-f285-4da1-bf0b-ebd18e4f6ea6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:47.595130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cabb2-f285-4da1-bf0b-ebd18e4f6ea6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:47.597650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8cabb2-f285-4da1-bf0b-ebd18e4f6ea6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-fc8cabb2-f285-4da1-bf0b-ebd18e4f6ea6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:47.622003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:47.622949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:47.623937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:47.623937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:47.623937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:47.623937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:47.629887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 422/842] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:47.633178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264cf20-b8af-4afa-9ca7-3fb339f614a3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:47.768505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:47.786487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2264cf20-b8af-4afa-9ca7-3fb339f614a3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:47.822962 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41c029-1ddb-4926-b93c-7f5a4f1aae04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:47.823975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41c029-1ddb-4926-b93c-7f5a4f1aae04 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:47.824506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41c029-1ddb-4926-b93c-7f5a4f1aae04 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:47.824979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41c029-1ddb-4926-b93c-7f5a4f1aae04 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:47.825916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41c029-1ddb-4926-b93c-7f5a4f1aae04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:47.826419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41c029-1ddb-4926-b93c-7f5a4f1aae04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:47.828084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41c029-1ddb-4926-b93c-7f5a4f1aae04 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:47.842026 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 421/843] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:47.852048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cabb2-f285-4da1-bf0b-ebd18e4f6ea6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:47.960779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8cabb2-f285-4da1-bf0b-ebd18e4f6ea6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:47.997382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aecd92-a31f-4b6d-a6fc-3246c4e6b926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:47.998547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aecd92-a31f-4b6d-a6fc-3246c4e6b926 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:47.999049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aecd92-a31f-4b6d-a6fc-3246c4e6b926 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:47.999511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aecd92-a31f-4b6d-a6fc-3246c4e6b926 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:48.008640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aecd92-a31f-4b6d-a6fc-3246c4e6b926 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:48.010481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aecd92-a31f-4b6d-a6fc-3246c4e6b926 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:48.014368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-39aecd92-a31f-4b6d-a6fc-3246c4e6b926 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:48.029047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:48.031364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:48.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 423/844] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:12:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:48.038085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41c029-1ddb-4926-b93c-7f5a4f1aae04 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:48.177146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41c029-1ddb-4926-b93c-7f5a4f1aae04 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:48.217397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf2346-934e-4d3d-bf99-c0d900784fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:48.218488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf2346-934e-4d3d-bf99-c0d900784fad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:48.218964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf2346-934e-4d3d-bf99-c0d900784fad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:48.219443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf2346-934e-4d3d-bf99-c0d900784fad None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:48.220702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf2346-934e-4d3d-bf99-c0d900784fad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:48.221219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf2346-934e-4d3d-bf99-c0d900784fad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:48.222909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf2346-934e-4d3d-bf99-c0d900784fad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:48.244047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 422/845] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Jul 26 07:12:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:48.253316 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aecd92-a31f-4b6d-a6fc-3246c4e6b926 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:48.403518 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aecd92-a31f-4b6d-a6fc-3246c4e6b926 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:48.438377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379ffe7-57f4-4c29-9df7-91c9d19a6920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:48.438951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379ffe7-57f4-4c29-9df7-91c9d19a6920 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:48.439231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379ffe7-57f4-4c29-9df7-91c9d19a6920 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:48.439509 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379ffe7-57f4-4c29-9df7-91c9d19a6920 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:48.512093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:48.514426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379ffe7-57f4-4c29-9df7-91c9d19a6920 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:48.515892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379ffe7-57f4-4c29-9df7-91c9d19a6920 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:48.619998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 424/846] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:12:48.626342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf2346-934e-4d3d-bf99-c0d900784fad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:48.636243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:48.701620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1c95b-b0ac-467c-88bf-e0af757b0ac5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:48.701848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1c95b-b0ac-467c-88bf-e0af757b0ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:48.702163 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d1c95b-b0ac-467c-88bf-e0af757b0ac5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:48.703748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 425/847] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:48.709486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e15945-6e95-4f70-995e-9e55807eeea4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:48.710002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e15945-6e95-4f70-995e-9e55807eeea4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:48.710474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e15945-6e95-4f70-995e-9e55807eeea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:48.729618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-28e15945-6e95-4f70-995e-9e55807eeea4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:48.730850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-28e15945-6e95-4f70-995e-9e55807eeea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:48.753730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf2346-934e-4d3d-bf99-c0d900784fad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:48.801835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fe236-7e92-43dd-bcf5-eee291c93682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:48.803594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fe236-7e92-43dd-bcf5-eee291c93682 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:48.804081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fe236-7e92-43dd-bcf5-eee291c93682 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:48.804629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fe236-7e92-43dd-bcf5-eee291c93682 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:48.814187 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 426/848] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:12:48.816502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fe236-7e92-43dd-bcf5-eee291c93682 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:48.818236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fe236-7e92-43dd-bcf5-eee291c93682 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:48.828273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e15945-6e95-4f70-995e-9e55807eeea4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-585fe236-7e92-43dd-bcf5-eee291c93682 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:48.839455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:48.840465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:48.841340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:48.841340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:48.841340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:48.841340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:48.844384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 423/849] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:48.852791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fe236-7e92-43dd-bcf5-eee291c93682 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:48.873459 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:48.924602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:48.967246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e15945-6e95-4f70-995e-9e55807eeea4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:48.975093 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fe236-7e92-43dd-bcf5-eee291c93682 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:49.012886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cceeaa2-9a75-4e24-8bb5-b7b04ff7a0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:49.013539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceeaa2-9a75-4e24-8bb5-b7b04ff7a0f9 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.013843 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceeaa2-9a75-4e24-8bb5-b7b04ff7a0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.014123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceeaa2-9a75-4e24-8bb5-b7b04ff7a0f9 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.014587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3169ca-acaa-4412-953b-989e5b145006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:49.014851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cceeaa2-9a75-4e24-8bb5-b7b04ff7a0f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:49.015676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3169ca-acaa-4412-953b-989e5b145006 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.016155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3169ca-acaa-4412-953b-989e5b145006 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.016259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cceeaa2-9a75-4e24-8bb5-b7b04ff7a0f9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:49.016734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3169ca-acaa-4412-953b-989e5b145006 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.017696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3169ca-acaa-4412-953b-989e5b145006 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:49.018228 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3169ca-acaa-4412-953b-989e5b145006 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:49.020034 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3169ca-acaa-4412-953b-989e5b145006 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:49.039059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 427/850] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:12:48 2022] POST /identity/v3/services => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:12:49.039267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 424/851] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:49.047477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cceeaa2-9a75-4e24-8bb5-b7b04ff7a0f9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:49.051549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e2c44-5d25-4861-aab3-d334b2a51cd3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.051864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e2c44-5d25-4861-aab3-d334b2a51cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.052115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829e2c44-5d25-4861-aab3-d334b2a51cd3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.053895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 425/852] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:49.059727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f94f1-0b9e-4d10-ad89-0cf3fe37b487 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.060043 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f94f1-0b9e-4d10-ad89-0cf3fe37b487 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.060284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f94f1-0b9e-4d10-ad89-0cf3fe37b487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.080109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-264f94f1-0b9e-4d10-ad89-0cf3fe37b487 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:49.081456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-264f94f1-0b9e-4d10-ad89-0cf3fe37b487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:49.172534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:49.187145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 426/853] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:12:49.189898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cceeaa2-9a75-4e24-8bb5-b7b04ff7a0f9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:49.211022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f94f1-0b9e-4d10-ad89-0cf3fe37b487 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:49.238654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350d93e-d552-4ec9-ad07-0b4fc616406a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:49.239617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350d93e-d552-4ec9-ad07-0b4fc616406a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.239921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350d93e-d552-4ec9-ad07-0b4fc616406a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.240202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5350d93e-d552-4ec9-ad07-0b4fc616406a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.248942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350d93e-d552-4ec9-ad07-0b4fc616406a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:49.250485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5350d93e-d552-4ec9-ad07-0b4fc616406a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5350d93e-d552-4ec9-ad07-0b4fc616406a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:49.264681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:49.265647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:49.266778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:49.266778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:49.266778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:49.266778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:49.266778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:49.266778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:49.266778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:49.266778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:49.268249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 428/854] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:49.272277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350d93e-d552-4ec9-ad07-0b4fc616406a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:49.276330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:49.340007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f94f1-0b9e-4d10-ad89-0cf3fe37b487 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:49.386324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86a118-dddc-408b-9a49-185fec933beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:49.387495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86a118-dddc-408b-9a49-185fec933beb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.387825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86a118-dddc-408b-9a49-185fec933beb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.388105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86a118-dddc-408b-9a49-185fec933beb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.404061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86a118-dddc-408b-9a49-185fec933beb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:49.404727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5350d93e-d552-4ec9-ad07-0b4fc616406a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2f86a118-dddc-408b-9a49-185fec933beb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:49.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:49.411936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:49.413245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 427/855] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:12:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:49.418806 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86a118-dddc-408b-9a49-185fec933beb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:49.461614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d845ff-595f-4690-a481-1bf27bb8668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:49.462371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d845ff-595f-4690-a481-1bf27bb8668b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.462656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d845ff-595f-4690-a481-1bf27bb8668b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.462929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d845ff-595f-4690-a481-1bf27bb8668b None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.463826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d845ff-595f-4690-a481-1bf27bb8668b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:49.464086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d845ff-595f-4690-a481-1bf27bb8668b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:49.465832 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d845ff-595f-4690-a481-1bf27bb8668b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:49.485884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 429/856] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:49.487649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d845ff-595f-4690-a481-1bf27bb8668b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:49.500631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:49.541611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86a118-dddc-408b-9a49-185fec933beb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:49.544500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:49.579840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3007628-cdb4-4ea8-8e3f-6333fa21f988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:49.580484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3007628-cdb4-4ea8-8e3f-6333fa21f988 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.580758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3007628-cdb4-4ea8-8e3f-6333fa21f988 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.581032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3007628-cdb4-4ea8-8e3f-6333fa21f988 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.582222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3007628-cdb4-4ea8-8e3f-6333fa21f988 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:49.582528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3007628-cdb4-4ea8-8e3f-6333fa21f988 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:49.583909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3007628-cdb4-4ea8-8e3f-6333fa21f988 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:49.592281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 428/857] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:12:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:49.593532 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83c28e-8c90-4593-89f3-d56596336ac9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.593846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83c28e-8c90-4593-89f3-d56596336ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.594128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea83c28e-8c90-4593-89f3-d56596336ac9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.595515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 429/858] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:49.598908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83c28e-8c90-4593-89f3-d56596336ac9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:49.602803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d845ff-595f-4690-a481-1bf27bb8668b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:49.638629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa0c1a-5a75-4da0-a22b-5ddd0b86c9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:49.639254 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa0c1a-5a75-4da0-a22b-5ddd0b86c9ef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.639635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa0c1a-5a75-4da0-a22b-5ddd0b86c9ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.639822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa0c1a-5a75-4da0-a22b-5ddd0b86c9ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.640578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa0c1a-5a75-4da0-a22b-5ddd0b86c9ef None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=64257d34b2df4af1a2016d56be8d44a8) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:49.640880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa0c1a-5a75-4da0-a22b-5ddd0b86c9ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:49.642345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa0c1a-5a75-4da0-a22b-5ddd0b86c9ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:49.648379 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 430/859] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:49 2022] GET /identity/v3/role_assignments?user.id=64257d34b2df4af1a2016d56be8d44a8&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:49.649617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c20d9-245d-4025-83bb-c0e5ac477a04 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.649926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c20d9-245d-4025-83bb-c0e5ac477a04 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.650198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c20d9-245d-4025-83bb-c0e5ac477a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.668790 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cc6c20d9-245d-4025-83bb-c0e5ac477a04 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:49.670127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cc6c20d9-245d-4025-83bb-c0e5ac477a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:49.744081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea83c28e-8c90-4593-89f3-d56596336ac9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:49.765389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 431/860] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:49.787055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c20d9-245d-4025-83bb-c0e5ac477a04 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:49.790773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:49.797189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808aed5d-2165-4e03-855f-64186724ac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:49.797909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808aed5d-2165-4e03-855f-64186724ac08 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.798192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808aed5d-2165-4e03-855f-64186724ac08 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.798462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808aed5d-2165-4e03-855f-64186724ac08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.813146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808aed5d-2165-4e03-855f-64186724ac08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:49.814720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808aed5d-2165-4e03-855f-64186724ac08 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-808aed5d-2165-4e03-855f-64186724ac08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:49.832259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:49.833200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:49.834118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:49.834118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:49.834118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:49.834118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:49.836267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 430/861] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:49.844550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808aed5d-2165-4e03-855f-64186724ac08 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:49.875763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:49.926622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c20d9-245d-4025-83bb-c0e5ac477a04 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:49.973758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b071-c222-46f0-a8fb-b3fadf54b1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:49.974440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b071-c222-46f0-a8fb-b3fadf54b1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:49.974440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b071-c222-46f0-a8fb-b3fadf54b1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:49.974758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80b071-c222-46f0-a8fb-b3fadf54b1a1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:49.975327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80b071-c222-46f0-a8fb-b3fadf54b1a1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:49.976833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80b071-c222-46f0-a8fb-b3fadf54b1a1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:49.985797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808aed5d-2165-4e03-855f-64186724ac08 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4b80b071-c222-46f0-a8fb-b3fadf54b1a1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:49.986947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:49.987981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 07:12:49.988827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:49.993293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 432/862] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Jul 26 07:12:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:49.994591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b071-c222-46f0-a8fb-b3fadf54b1a1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:50.029940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988b221-f4b0-4e12-bb1c-1dab5e430d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:50.030480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988b221-f4b0-4e12-bb1c-1dab5e430d98 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:50.030759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988b221-f4b0-4e12-bb1c-1dab5e430d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:50.031034 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3988b221-f4b0-4e12-bb1c-1dab5e430d98 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:50.031886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988b221-f4b0-4e12-bb1c-1dab5e430d98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:50.032189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988b221-f4b0-4e12-bb1c-1dab5e430d98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:50.033559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3988b221-f4b0-4e12-bb1c-1dab5e430d98 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:50.047921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 431/863] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:50.053628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988b221-f4b0-4e12-bb1c-1dab5e430d98 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:50.093685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:50.139399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80b071-c222-46f0-a8fb-b3fadf54b1a1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:50.155164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:50.182133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3988b221-f4b0-4e12-bb1c-1dab5e430d98 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:50.185033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08afdea-cd0e-4dc4-b93d-573a8d768e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:50.186279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08afdea-cd0e-4dc4-b93d-573a8d768e4c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:50.186279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08afdea-cd0e-4dc4-b93d-573a8d768e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:50.186559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08afdea-cd0e-4dc4-b93d-573a8d768e4c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:50.187056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08afdea-cd0e-4dc4-b93d-573a8d768e4c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:50.187378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08afdea-cd0e-4dc4-b93d-573a8d768e4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:50.188926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08afdea-cd0e-4dc4-b93d-573a8d768e4c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:50.199199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 433/864] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:12:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:50.200734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08afdea-cd0e-4dc4-b93d-573a8d768e4c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:50.230697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173148da-a112-4dbb-9b62-cfd638fa0ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:50.232249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173148da-a112-4dbb-9b62-cfd638fa0ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:50.232878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173148da-a112-4dbb-9b62-cfd638fa0ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:50.233220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173148da-a112-4dbb-9b62-cfd638fa0ee7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:50.241847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173148da-a112-4dbb-9b62-cfd638fa0ee7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:50.243302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173148da-a112-4dbb-9b62-cfd638fa0ee7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-173148da-a112-4dbb-9b62-cfd638fa0ee7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:50.255951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:50.256951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:50.257910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 432/865] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jul 26 07:12:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:50.266827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173148da-a112-4dbb-9b62-cfd638fa0ee7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:50.329669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08afdea-cd0e-4dc4-b93d-573a8d768e4c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:50.361127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:50.368429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84780f59-d8ad-4d4d-9fda-b6030ebbad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:50.369400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84780f59-d8ad-4d4d-9fda-b6030ebbad01 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:50.369895 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84780f59-d8ad-4d4d-9fda-b6030ebbad01 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:50.370349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84780f59-d8ad-4d4d-9fda-b6030ebbad01 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:50.371213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84780f59-d8ad-4d4d-9fda-b6030ebbad01 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:50.371697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84780f59-d8ad-4d4d-9fda-b6030ebbad01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:50.373303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84780f59-d8ad-4d4d-9fda-b6030ebbad01 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:50.379019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 434/866] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:12:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:50.384706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173148da-a112-4dbb-9b62-cfd638fa0ee7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:50.425529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef8d8b-0af8-434c-aade-2dfa8d6e411e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:50.426540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef8d8b-0af8-434c-aade-2dfa8d6e411e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:50.427058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef8d8b-0af8-434c-aade-2dfa8d6e411e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:50.427517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef8d8b-0af8-434c-aade-2dfa8d6e411e None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:50.428517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef8d8b-0af8-434c-aade-2dfa8d6e411e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:50.429080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef8d8b-0af8-434c-aade-2dfa8d6e411e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:50.430790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef8d8b-0af8-434c-aade-2dfa8d6e411e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:50.445798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:50.452127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 433/867] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jul 26 07:12:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:50.458201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84780f59-d8ad-4d4d-9fda-b6030ebbad01 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:50.572252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84780f59-d8ad-4d4d-9fda-b6030ebbad01 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:50.608548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c3e34-9f8a-49b6-99cf-f4b4df5973dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:50.609470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c3e34-9f8a-49b6-99cf-f4b4df5973dc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:50.609791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c3e34-9f8a-49b6-99cf-f4b4df5973dc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:50.610062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c3e34-9f8a-49b6-99cf-f4b4df5973dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:50.676206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:50.683784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c3e34-9f8a-49b6-99cf-f4b4df5973dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:50.685575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c3e34-9f8a-49b6-99cf-f4b4df5973dc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:50.789661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 435/868] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:50 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:12:50.801799 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef8d8b-0af8-434c-aade-2dfa8d6e411e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:50.903490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef8d8b-0af8-434c-aade-2dfa8d6e411e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:50.944040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617f87b-1097-4341-871a-de973969948f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:50.944711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617f87b-1097-4341-871a-de973969948f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:50.944994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617f87b-1097-4341-871a-de973969948f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:50.945272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617f87b-1097-4341-871a-de973969948f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:50.957283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617f87b-1097-4341-871a-de973969948f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:50.958877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617f87b-1097-4341-871a-de973969948f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-2617f87b-1097-4341-871a-de973969948f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:50.975571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:50.976545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:50.978192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:50.978192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:50.978192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:50.978192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:50.978192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 434/869] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:50.983021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c3e34-9f8a-49b6-99cf-f4b4df5973dc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:51.100539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:51.106526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c3e34-9f8a-49b6-99cf-f4b4df5973dc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:51.123902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d52df-fc18-4d73-8369-243629d4575b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.124289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d52df-fc18-4d73-8369-243629d4575b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.124474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58d52df-fc18-4d73-8369-243629d4575b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.126082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 435/870] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:51.131526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cff3a-2b36-4746-bc7a-52165c69edec None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.131816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cff3a-2b36-4746-bc7a-52165c69edec None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.132091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589cff3a-2b36-4746-bc7a-52165c69edec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.143269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a149c6-3e87-4b39-b5d0-2ac4764b1f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:51.143906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a149c6-3e87-4b39-b5d0-2ac4764b1f6c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.144241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a149c6-3e87-4b39-b5d0-2ac4764b1f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.144516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a149c6-3e87-4b39-b5d0-2ac4764b1f6c None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.145216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a149c6-3e87-4b39-b5d0-2ac4764b1f6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:51.145519 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a149c6-3e87-4b39-b5d0-2ac4764b1f6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:51.146923 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a149c6-3e87-4b39-b5d0-2ac4764b1f6c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:51.151589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-589cff3a-2b36-4746-bc7a-52165c69edec None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:51.152787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-589cff3a-2b36-4746-bc7a-52165c69edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:51.159570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 436/871] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:51.164711 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a149c6-3e87-4b39-b5d0-2ac4764b1f6c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:51.222186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:51.237434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 436/872] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:51.262854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cff3a-2b36-4746-bc7a-52165c69edec None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:51.271691 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a149c6-3e87-4b39-b5d0-2ac4764b1f6c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:51.307263 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172da9bd-baaa-4de4-b387-7c511e1bcd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:51.307935 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172da9bd-baaa-4de4-b387-7c511e1bcd73 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.308217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172da9bd-baaa-4de4-b387-7c511e1bcd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.308492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172da9bd-baaa-4de4-b387-7c511e1bcd73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.315659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172da9bd-baaa-4de4-b387-7c511e1bcd73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:51.317184 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172da9bd-baaa-4de4-b387-7c511e1bcd73 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-172da9bd-baaa-4de4-b387-7c511e1bcd73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:51.329248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:51.330222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:51.331100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:51.331100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:51.331100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:51.331100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:51.331100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:51.331100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:51.331100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:51.331100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:51.335762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 437/873] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:51.336682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172da9bd-baaa-4de4-b387-7c511e1bcd73 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:51.356449 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:51.362798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589cff3a-2b36-4746-bc7a-52165c69edec None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:51.398920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152b105-059c-444d-b884-65529c47dbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:51.399581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152b105-059c-444d-b884-65529c47dbbf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.399865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152b105-059c-444d-b884-65529c47dbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.400143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1152b105-059c-444d-b884-65529c47dbbf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.400895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152b105-059c-444d-b884-65529c47dbbf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:51.402421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1152b105-059c-444d-b884-65529c47dbbf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1152b105-059c-444d-b884-65529c47dbbf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:51.412377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:51.413662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:51.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 437/874] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:51.419653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152b105-059c-444d-b884-65529c47dbbf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:51.460389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172da9bd-baaa-4de4-b387-7c511e1bcd73 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:51.492334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:51.498969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a50a7-48ba-423c-8d56-f7284079460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:51.499612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a50a7-48ba-423c-8d56-f7284079460b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.499896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a50a7-48ba-423c-8d56-f7284079460b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.500170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240a50a7-48ba-423c-8d56-f7284079460b None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.500928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a50a7-48ba-423c-8d56-f7284079460b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:51.501238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a50a7-48ba-423c-8d56-f7284079460b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:51.502930 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240a50a7-48ba-423c-8d56-f7284079460b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:51.521483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 438/875] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:51.522530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a50a7-48ba-423c-8d56-f7284079460b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:51.566772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1152b105-059c-444d-b884-65529c47dbbf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:51.611544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312880e-396a-4f20-bca6-e8600d49b9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:51.612216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312880e-396a-4f20-bca6-e8600d49b9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.612644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312880e-396a-4f20-bca6-e8600d49b9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.613244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312880e-396a-4f20-bca6-e8600d49b9b4 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.613994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312880e-396a-4f20-bca6-e8600d49b9b4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:51.614298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312880e-396a-4f20-bca6-e8600d49b9b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:51.615863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312880e-396a-4f20-bca6-e8600d49b9b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:51.622832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 438/876] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:51.625113 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:51.627595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312880e-396a-4f20-bca6-e8600d49b9b4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:51.656264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240a50a7-48ba-423c-8d56-f7284079460b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:51.705513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5df13-e540-459f-aa3e-4d0ae69090ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:51.706776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5df13-e540-459f-aa3e-4d0ae69090ef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.707934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5df13-e540-459f-aa3e-4d0ae69090ef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.708418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5df13-e540-459f-aa3e-4d0ae69090ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.709464 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5df13-e540-459f-aa3e-4d0ae69090ef None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=77fb7ab7232b48a286b7798beca19f58) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:51.710082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5df13-e540-459f-aa3e-4d0ae69090ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:51.711964 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5df13-e540-459f-aa3e-4d0ae69090ef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:51.720294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 439/877] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:51 2022] GET /identity/v3/role_assignments?user.id=77fb7ab7232b48a286b7798beca19f58&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:51.756513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312880e-396a-4f20-bca6-e8600d49b9b4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:51.764746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:51.796535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47cd58-8d13-4dde-86ac-d67ad9170f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:51.797244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47cd58-8d13-4dde-86ac-d67ad9170f71 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.797531 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47cd58-8d13-4dde-86ac-d67ad9170f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.797832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47cd58-8d13-4dde-86ac-d67ad9170f71 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.798616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47cd58-8d13-4dde-86ac-d67ad9170f71 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:51.798910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47cd58-8d13-4dde-86ac-d67ad9170f71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:51.800413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47cd58-8d13-4dde-86ac-d67ad9170f71 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:51.806553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 439/878] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:51.812418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5df13-e540-459f-aa3e-4d0ae69090ef None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:51.849484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc0801-c028-4bfd-8236-225a9c4f16c8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.849791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc0801-c028-4bfd-8236-225a9c4f16c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.850135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfc0801-c028-4bfd-8236-225a9c4f16c8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.851981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 440/879] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:12:51.859688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e173085-815a-49a3-ab9b-ef653fadb87b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.859975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e173085-815a-49a3-ab9b-ef653fadb87b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.860251 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e173085-815a-49a3-ab9b-ef653fadb87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.880381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7e173085-815a-49a3-ab9b-ef653fadb87b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:51.881670 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e173085-815a-49a3-ab9b-ef653fadb87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:51.943561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5df13-e540-459f-aa3e-4d0ae69090ef None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:51.983068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 441/880] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:51 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:12:51.989704 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc7e4a-d8c8-4ea9-ac9a-fd0b93cfbcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:51.990415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc7e4a-d8c8-4ea9-ac9a-fd0b93cfbcc8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:51.990697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc7e4a-d8c8-4ea9-ac9a-fd0b93cfbcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:51.990969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bc7e4a-d8c8-4ea9-ac9a-fd0b93cfbcc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:51.991743 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc7e4a-d8c8-4ea9-ac9a-fd0b93cfbcc8 None admin] RBAC: Adding query filter params (interface=public, service_id=47ea4d3b280b4c79a1e7f6996ba513e4, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:51.992053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc7e4a-d8c8-4ea9-ac9a-fd0b93cfbcc8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:51.993435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bc7e4a-d8c8-4ea9-ac9a-fd0b93cfbcc8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:51.999446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 440/881] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jul 26 07:12:51 2022] GET /identity/v3/endpoints?service_id=47ea4d3b280b4c79a1e7f6996ba513e4&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:52.004975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc7e4a-d8c8-4ea9-ac9a-fd0b93cfbcc8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:52.005197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e173085-815a-49a3-ab9b-ef653fadb87b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:52.010308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:52.110190 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:52.116842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e173085-815a-49a3-ab9b-ef653fadb87b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:52.123012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bc7e4a-d8c8-4ea9-ac9a-fd0b93cfbcc8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:52.160710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2bd50b-5c74-4456-9661-a29d922cb898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:52.161429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bd50b-5c74-4456-9661-a29d922cb898 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:52.161705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bd50b-5c74-4456-9661-a29d922cb898 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:52.162022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2bd50b-5c74-4456-9661-a29d922cb898 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:52.162655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2bd50b-5c74-4456-9661-a29d922cb898 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:52.164057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2bd50b-5c74-4456-9661-a29d922cb898 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:52.164497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe9ebe-e267-402a-9af7-20bd04836e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:52.165336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe9ebe-e267-402a-9af7-20bd04836e61 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:52.166124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe9ebe-e267-402a-9af7-20bd04836e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:52.166592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfe9ebe-e267-402a-9af7-20bd04836e61 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:52.167819 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe9ebe-e267-402a-9af7-20bd04836e61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:52.169641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfe9ebe-e267-402a-9af7-20bd04836e61 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4e2bd50b-5c74-4456-9661-a29d922cb898 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:52.174019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:52.175018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:52.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 441/882] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:12:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:52.182152 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 442/883] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jul 26 07:12:52 2022] GET /identity/v3/services => generated 1011 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:12:52.189784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2bd50b-5c74-4456-9661-a29d922cb898 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:52.262915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:52.308100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2bd50b-5c74-4456-9661-a29d922cb898 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:52.349430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a4999-d583-4dcc-a216-8b7a313b3d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:52.350366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a4999-d583-4dcc-a216-8b7a313b3d69 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:52.350784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a4999-d583-4dcc-a216-8b7a313b3d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:52.351193 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a4999-d583-4dcc-a216-8b7a313b3d69 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:52.352025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a4999-d583-4dcc-a216-8b7a313b3d69 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:52.352517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a4999-d583-4dcc-a216-8b7a313b3d69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:52.354091 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171a4999-d583-4dcc-a216-8b7a313b3d69 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:52.360539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 442/884] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jul 26 07:12:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:52.368222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe9ebe-e267-402a-9af7-20bd04836e61 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:52.485576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfe9ebe-e267-402a-9af7-20bd04836e61 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:52.521270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace510f-583c-433c-a4eb-479406669af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:52.521846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace510f-583c-433c-a4eb-479406669af9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:52.522126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace510f-583c-433c-a4eb-479406669af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:52.522400 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace510f-583c-433c-a4eb-479406669af9 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:52.523092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace510f-583c-433c-a4eb-479406669af9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:52.523392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace510f-583c-433c-a4eb-479406669af9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:52.524770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace510f-583c-433c-a4eb-479406669af9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:52.531415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 443/885] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jul 26 07:12:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:52.577338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449742f-bf79-4797-b1f1-36df4799bba5 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:52.577672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449742f-bf79-4797-b1f1-36df4799bba5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:52.577911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449742f-bf79-4797-b1f1-36df4799bba5 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:52.579525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 443/886] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:52.588704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8d2f4-5a58-4225-841b-88341f2b0c82 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:52.588704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8d2f4-5a58-4225-841b-88341f2b0c82 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:52.588704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae8d2f4-5a58-4225-841b-88341f2b0c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:52.606337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8ae8d2f4-5a58-4225-841b-88341f2b0c82 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:52.607862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ae8d2f4-5a58-4225-841b-88341f2b0c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:52.636214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:52.686760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 444/887] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:52 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:52.710747 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449742f-bf79-4797-b1f1-36df4799bba5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:52.822841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449742f-bf79-4797-b1f1-36df4799bba5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:52.853037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:52.859410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f434b099-a93c-4b6a-a939-3dc804c18e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:52.860342 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434b099-a93c-4b6a-a939-3dc804c18e71 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:52.860807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434b099-a93c-4b6a-a939-3dc804c18e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:52.861258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f434b099-a93c-4b6a-a939-3dc804c18e71 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:52.862302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434b099-a93c-4b6a-a939-3dc804c18e71 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:52.863950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f434b099-a93c-4b6a-a939-3dc804c18e71 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:52.884469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 444/888] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:12:52 2022] POST /identity/v3/services => generated 269 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:12:54.294854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6ae3b-2cdc-4501-882c-57ff5c78aec8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:54.295791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6ae3b-2cdc-4501-882c-57ff5c78aec8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:54.295791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a6ae3b-2cdc-4501-882c-57ff5c78aec8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:54.297040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 445/889] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:54.302477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a94a5-a660-4515-9ee0-bb8d544f09ec None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:54.303360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a94a5-a660-4515-9ee0-bb8d544f09ec None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:54.303859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697a94a5-a660-4515-9ee0-bb8d544f09ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:54.322512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-697a94a5-a660-4515-9ee0-bb8d544f09ec None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:54.324068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-697a94a5-a660-4515-9ee0-bb8d544f09ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:54.381760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310218-0082-415a-b5a5-845b42fd21f4 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:54.382755 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310218-0082-415a-b5a5-845b42fd21f4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:54.383566 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13310218-0082-415a-b5a5-845b42fd21f4 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:54.393219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 446/890] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:54.396427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74e155-8c1f-46ec-81d8-1a1327674e35 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:54.397290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74e155-8c1f-46ec-81d8-1a1327674e35 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:54.398114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74e155-8c1f-46ec-81d8-1a1327674e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:54.406115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 445/891] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:54.426116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1d74e155-8c1f-46ec-81d8-1a1327674e35 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:54.427751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d74e155-8c1f-46ec-81d8-1a1327674e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:54.433596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a94a5-a660-4515-9ee0-bb8d544f09ec None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:54.513892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 447/892] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:12:54.534631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74e155-8c1f-46ec-81d8-1a1327674e35 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:54.581881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697a94a5-a660-4515-9ee0-bb8d544f09ec None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:54.614331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:54.651778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3793d-a414-48fe-af23-2b3e91b675e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:54.653298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3793d-a414-48fe-af23-2b3e91b675e2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:54.654089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3793d-a414-48fe-af23-2b3e91b675e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:54.654871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3793d-a414-48fe-af23-2b3e91b675e2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:54.656322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3793d-a414-48fe-af23-2b3e91b675e2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:54.658521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:54.660628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa3793d-a414-48fe-af23-2b3e91b675e2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:54.672982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74e155-8c1f-46ec-81d8-1a1327674e35 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4aa3793d-a414-48fe-af23-2b3e91b675e2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:54.680286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:54.681611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 26 07:12:54.682591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:54.683387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 446/893] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:54.690769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3793d-a414-48fe-af23-2b3e91b675e2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:54.725571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05978015-e466-4556-ada9-fbf0868d860b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:54.726269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05978015-e466-4556-ada9-fbf0868d860b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:54.726556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05978015-e466-4556-ada9-fbf0868d860b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:54.726986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05978015-e466-4556-ada9-fbf0868d860b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:54.738495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05978015-e466-4556-ada9-fbf0868d860b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-05978015-e466-4556-ada9-fbf0868d860b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:54.743801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:54.744735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:54.745575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 448/894] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jul 26 07:12:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:54.746919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0570-769a-4baa-ab96-838ab649dab2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:54.746919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0570-769a-4baa-ab96-838ab649dab2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:54.746919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f0570-769a-4baa-ab96-838ab649dab2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:54.748183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 449/895] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:54.753327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf93fd-2fda-4996-9239-ee5eafd9e011 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:54.754654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf93fd-2fda-4996-9239-ee5eafd9e011 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:54.754654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf93fd-2fda-4996-9239-ee5eafd9e011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:54.774483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7baf93fd-2fda-4996-9239-ee5eafd9e011 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:54.776238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7baf93fd-2fda-4996-9239-ee5eafd9e011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:54.832870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3793d-a414-48fe-af23-2b3e91b675e2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:54.863761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 450/896] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:12:54.864739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf93fd-2fda-4996-9239-ee5eafd9e011 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:54.879179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f03789-acf2-4323-8c92-1ed495a27e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:54.880801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f03789-acf2-4323-8c92-1ed495a27e42 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:54.881609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f03789-acf2-4323-8c92-1ed495a27e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:54.882075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f03789-acf2-4323-8c92-1ed495a27e42 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:54.883686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f03789-acf2-4323-8c92-1ed495a27e42 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:54.884750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f03789-acf2-4323-8c92-1ed495a27e42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:54.886668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f03789-acf2-4323-8c92-1ed495a27e42 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:54.896003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 447/897] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:54.898886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f03789-acf2-4323-8c92-1ed495a27e42 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:54.949331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:54.951592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:54.975302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf93fd-2fda-4996-9239-ee5eafd9e011 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:55.008543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f03789-acf2-4323-8c92-1ed495a27e42 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:55.010782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cb5fd-566d-439d-9a7d-8f081424f8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:55.011506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb5fd-566d-439d-9a7d-8f081424f8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.011793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb5fd-566d-439d-9a7d-8f081424f8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.012070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8cb5fd-566d-439d-9a7d-8f081424f8f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.012800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cb5fd-566d-439d-9a7d-8f081424f8f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:55.013105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cb5fd-566d-439d-9a7d-8f081424f8f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:55.014545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8cb5fd-566d-439d-9a7d-8f081424f8f8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:55.023103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 451/898] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jul 26 07:12:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:55.023896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cb5fd-566d-439d-9a7d-8f081424f8f8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:55.041663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01fc4-f945-4617-8776-3a43cabe5371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:55.042331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01fc4-f945-4617-8776-3a43cabe5371 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.042610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01fc4-f945-4617-8776-3a43cabe5371 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.042883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01fc4-f945-4617-8776-3a43cabe5371 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.043573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c01fc4-f945-4617-8776-3a43cabe5371 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:55.044951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c01fc4-f945-4617-8776-3a43cabe5371 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:55.059967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 448/899] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:12:54 2022] POST /identity/v3/services => generated 241 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:55.061238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942ef36-cc1c-4259-9dce-3301ec2f00c8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.061525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942ef36-cc1c-4259-9dce-3301ec2f00c8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.061839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942ef36-cc1c-4259-9dce-3301ec2f00c8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.063367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 449/900] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:55.064115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942ef36-cc1c-4259-9dce-3301ec2f00c8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:55.143343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8cb5fd-566d-439d-9a7d-8f081424f8f8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:55.193260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40db3f2-63b6-4be5-87f2-2a5a28b74b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:55.194100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db3f2-63b6-4be5-87f2-2a5a28b74b0f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.194388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db3f2-63b6-4be5-87f2-2a5a28b74b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.194747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40db3f2-63b6-4be5-87f2-2a5a28b74b0f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.195476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40db3f2-63b6-4be5-87f2-2a5a28b74b0f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:55.195752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40db3f2-63b6-4be5-87f2-2a5a28b74b0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:55.197170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40db3f2-63b6-4be5-87f2-2a5a28b74b0f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:55.200085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:55.203518 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 452/901] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Jul 26 07:12:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:55.204771 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61237fb9-1ee9-4f42-960c-6a95e17eef3f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.205069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61237fb9-1ee9-4f42-960c-6a95e17eef3f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.205351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61237fb9-1ee9-4f42-960c-6a95e17eef3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.206186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942ef36-cc1c-4259-9dce-3301ec2f00c8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:55.217086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:55.226101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-61237fb9-1ee9-4f42-960c-6a95e17eef3f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:55.227000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-61237fb9-1ee9-4f42-960c-6a95e17eef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:55.257691 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb6f59-3358-4083-abce-bcbe08affbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:55.258457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb6f59-3358-4083-abce-bcbe08affbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.258738 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb6f59-3358-4083-abce-bcbe08affbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.259012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb6f59-3358-4083-abce-bcbe08affbf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.271690 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb6f59-3358-4083-abce-bcbe08affbf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:55.273250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abb6f59-3358-4083-abce-bcbe08affbf9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3abb6f59-3358-4083-abce-bcbe08affbf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:55.289167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:55.290260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:55.291144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:55.291144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:55.291144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:55.291144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:55.291144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 450/902] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:55.291995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb6f59-3358-4083-abce-bcbe08affbf9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:55.312033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 453/903] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:55.314251 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61237fb9-1ee9-4f42-960c-6a95e17eef3f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:55.403979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb6f59-3358-4083-abce-bcbe08affbf9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:55.421057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61237fb9-1ee9-4f42-960c-6a95e17eef3f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:55.448630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc328ed-3f83-4452-ba88-0ac19448c4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:55.449280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc328ed-3f83-4452-ba88-0ac19448c4d3 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.449562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc328ed-3f83-4452-ba88-0ac19448c4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.449877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc328ed-3f83-4452-ba88-0ac19448c4d3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.450597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc328ed-3f83-4452-ba88-0ac19448c4d3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:55.451375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:55.451961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc328ed-3f83-4452-ba88-0ac19448c4d3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:55.458423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4ea7a-3722-49a7-bbb6-b389d6f4cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:55.459075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4ea7a-3722-49a7-bbb6-b389d6f4cac1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.459414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4ea7a-3722-49a7-bbb6-b389d6f4cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.459625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4ea7a-3722-49a7-bbb6-b389d6f4cac1 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.460318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4ea7a-3722-49a7-bbb6-b389d6f4cac1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:55.460717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4ea7a-3722-49a7-bbb6-b389d6f4cac1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:55.462055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4ea7a-3722-49a7-bbb6-b389d6f4cac1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:55.481765 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 454/904] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:55.481991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 451/905] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:12:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:55.482098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4ea7a-3722-49a7-bbb6-b389d6f4cac1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:55.482186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc328ed-3f83-4452-ba88-0ac19448c4d3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:55.500229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:55.618387 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4ea7a-3722-49a7-bbb6-b389d6f4cac1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:55.641706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc328ed-3f83-4452-ba88-0ac19448c4d3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:55.658740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a62ef0-be11-4387-9113-472a2baf65c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:55.659298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62ef0-be11-4387-9113-472a2baf65c2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.659582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62ef0-be11-4387-9113-472a2baf65c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.659907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a62ef0-be11-4387-9113-472a2baf65c2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.660548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a62ef0-be11-4387-9113-472a2baf65c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:55.661975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a62ef0-be11-4387-9113-472a2baf65c2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-80a62ef0-be11-4387-9113-472a2baf65c2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:55.676036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:55.677033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:55.677913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 455/906] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Jul 26 07:12:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:55.683311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a62ef0-be11-4387-9113-472a2baf65c2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:55.690554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c3e69-9afe-45d9-9d6d-dbaa35ecac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:55.691544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c3e69-9afe-45d9-9d6d-dbaa35ecac47 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.692025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c3e69-9afe-45d9-9d6d-dbaa35ecac47 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.692298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c3e69-9afe-45d9-9d6d-dbaa35ecac47 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.701454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c3e69-9afe-45d9-9d6d-dbaa35ecac47 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:55.702994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c3e69-9afe-45d9-9d6d-dbaa35ecac47 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ad7c3e69-9afe-45d9-9d6d-dbaa35ecac47 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:12:55.716273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:55.717339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:55.718314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 452/907] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jul 26 07:12:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:55.723592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c3e69-9afe-45d9-9d6d-dbaa35ecac47 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:55.754091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:55.813522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a62ef0-be11-4387-9113-472a2baf65c2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:55.821196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:55.861270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d66f9d-2400-4e65-b027-b91a3df28ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:55.861947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c3e69-9afe-45d9-9d6d-dbaa35ecac47 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:55.862069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d66f9d-2400-4e65-b027-b91a3df28ded None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.862222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d66f9d-2400-4e65-b027-b91a3df28ded None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.862505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d66f9d-2400-4e65-b027-b91a3df28ded None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.863335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d66f9d-2400-4e65-b027-b91a3df28ded None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:55.863650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d66f9d-2400-4e65-b027-b91a3df28ded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:55.865347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d66f9d-2400-4e65-b027-b91a3df28ded None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:55.875929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 456/908] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:12:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:55.877197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d66f9d-2400-4e65-b027-b91a3df28ded None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:55.897878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18819380-fe15-4d97-8104-fc5698f87bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:55.898522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18819380-fe15-4d97-8104-fc5698f87bad None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:55.898803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18819380-fe15-4d97-8104-fc5698f87bad None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:55.899071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18819380-fe15-4d97-8104-fc5698f87bad None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:55.899997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18819380-fe15-4d97-8104-fc5698f87bad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:55.900302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18819380-fe15-4d97-8104-fc5698f87bad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:55.901768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18819380-fe15-4d97-8104-fc5698f87bad None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:55.924632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 453/909] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jul 26 07:12:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:55.926057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18819380-fe15-4d97-8104-fc5698f87bad None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:55.990196 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d66f9d-2400-4e65-b027-b91a3df28ded None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:56.009553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:56.028116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18819380-fe15-4d97-8104-fc5698f87bad None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:56.029155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27272e-9404-4dba-8977-32cfa38bdca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:56.029939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27272e-9404-4dba-8977-32cfa38bdca7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:56.030226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27272e-9404-4dba-8977-32cfa38bdca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:56.030603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27272e-9404-4dba-8977-32cfa38bdca7 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:56.031311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27272e-9404-4dba-8977-32cfa38bdca7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:56.031618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27272e-9404-4dba-8977-32cfa38bdca7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:56.033085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27272e-9404-4dba-8977-32cfa38bdca7 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:56.044251 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 457/910] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:12:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:12:56.045312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27272e-9404-4dba-8977-32cfa38bdca7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:56.059544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:56.069753 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaf4ff-a183-47e9-8542-c464d5add1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:56.070527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf4ff-a183-47e9-8542-c464d5add1fa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:56.070880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf4ff-a183-47e9-8542-c464d5add1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:56.071223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaf4ff-a183-47e9-8542-c464d5add1fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:56.143342 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaf4ff-a183-47e9-8542-c464d5add1fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:56.145079 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaf4ff-a183-47e9-8542-c464d5add1fa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:56.165111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27272e-9404-4dba-8977-32cfa38bdca7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:56.203838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171312f-568d-4e07-894b-b607d2d3e2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:56.204544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171312f-568d-4e07-894b-b607d2d3e2d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:56.204832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171312f-568d-4e07-894b-b607d2d3e2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:56.205109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171312f-568d-4e07-894b-b607d2d3e2d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:56.206071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171312f-568d-4e07-894b-b607d2d3e2d0 None admin] RBAC: Adding query filter params (interface=public, service_id=39c1dc51cd7441f58f4174014511112d, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:56.206385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171312f-568d-4e07-894b-b607d2d3e2d0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:56.207889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171312f-568d-4e07-894b-b607d2d3e2d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:56.219281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 458/911] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jul 26 07:12:56 2022] GET /identity/v3/endpoints?service_id=39c1dc51cd7441f58f4174014511112d&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:56.220504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171312f-568d-4e07-894b-b607d2d3e2d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:56.255913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 454/912] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:12:55 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:12:56.257271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaf4ff-a183-47e9-8542-c464d5add1fa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:56.273443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:56.286469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:56.334187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171312f-568d-4e07-894b-b607d2d3e2d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:56.365375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaf4ff-a183-47e9-8542-c464d5add1fa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:56.371240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90452045-6faf-428f-a307-8a8105b8c94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:56.371878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90452045-6faf-428f-a307-8a8105b8c94f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:56.372161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90452045-6faf-428f-a307-8a8105b8c94f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:56.372426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90452045-6faf-428f-a307-8a8105b8c94f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:56.373121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90452045-6faf-428f-a307-8a8105b8c94f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:56.374593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90452045-6faf-428f-a307-8a8105b8c94f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:56.390236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 459/913] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jul 26 07:12:56 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:12:56.401916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a8757-18a1-4e95-b2e1-fb69e205cd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:56.402627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a8757-18a1-4e95-b2e1-fb69e205cd0c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:56.402907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a8757-18a1-4e95-b2e1-fb69e205cd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:56.403181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a8757-18a1-4e95-b2e1-fb69e205cd0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:56.415270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a8757-18a1-4e95-b2e1-fb69e205cd0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:56.416838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a8757-18a1-4e95-b2e1-fb69e205cd0c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-792a8757-18a1-4e95-b2e1-fb69e205cd0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:56.433607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:56.435723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:56.437786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:56.437786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:56.437786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:56.437786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:56.437786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 455/914] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:56.443416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90452045-6faf-428f-a307-8a8105b8c94f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:56.556592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90452045-6faf-428f-a307-8a8105b8c94f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:56.569312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:56.593399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a5d86-2c30-4dd3-90dc-239a2a486782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:56.594418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a5d86-2c30-4dd3-90dc-239a2a486782 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:56.594922 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a5d86-2c30-4dd3-90dc-239a2a486782 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:56.595413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a5d86-2c30-4dd3-90dc-239a2a486782 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:56.596491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a5d86-2c30-4dd3-90dc-239a2a486782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:56.597003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a5d86-2c30-4dd3-90dc-239a2a486782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:56.598654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9a5d86-2c30-4dd3-90dc-239a2a486782 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:56.612708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 460/915] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:56.618235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a8757-18a1-4e95-b2e1-fb69e205cd0c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:56.702404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:56.726481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a8757-18a1-4e95-b2e1-fb69e205cd0c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:56.762563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d849c-3c8f-4cc8-a1ce-529d3c2ea852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:56.763224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d849c-3c8f-4cc8-a1ce-529d3c2ea852 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:56.763504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d849c-3c8f-4cc8-a1ce-529d3c2ea852 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:56.763778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537d849c-3c8f-4cc8-a1ce-529d3c2ea852 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:56.771173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d849c-3c8f-4cc8-a1ce-529d3c2ea852 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:56.772650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537d849c-3c8f-4cc8-a1ce-529d3c2ea852 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-537d849c-3c8f-4cc8-a1ce-529d3c2ea852 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:56.784488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:56.785532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:56.786463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:56.786463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:56.786463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:56.786463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:56.786463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:56.786463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:56.786463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:56.786463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:56.786463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 456/916] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:56.791702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a5d86-2c30-4dd3-90dc-239a2a486782 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:56.893474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a5d86-2c30-4dd3-90dc-239a2a486782 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:56.932716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3460336-668f-49d8-939b-762366cf7d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:56.933778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3460336-668f-49d8-939b-762366cf7d2f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:56.934284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3460336-668f-49d8-939b-762366cf7d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:56.934778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3460336-668f-49d8-939b-762366cf7d2f None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:56.935731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3460336-668f-49d8-939b-762366cf7d2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:56.936248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3460336-668f-49d8-939b-762366cf7d2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:56.937919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3460336-668f-49d8-939b-762366cf7d2f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:56.950516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 461/917] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:12:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:56.961227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d849c-3c8f-4cc8-a1ce-529d3c2ea852 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:57.069666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537d849c-3c8f-4cc8-a1ce-529d3c2ea852 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:57.103555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d5cdce-db94-457d-a41e-a169edbb0752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:57.104190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5cdce-db94-457d-a41e-a169edbb0752 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:57.104469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5cdce-db94-457d-a41e-a169edbb0752 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:57.104740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d5cdce-db94-457d-a41e-a169edbb0752 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:57.105536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5cdce-db94-457d-a41e-a169edbb0752 None admin] RBAC: Adding query filter params (role.id=06dfeb3edc3b4716b86a7f7be42d71b1, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=77fb7ab7232b48a286b7798beca19f58) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:57.105870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5cdce-db94-457d-a41e-a169edbb0752 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:57.107315 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d5cdce-db94-457d-a41e-a169edbb0752 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:57.113871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 457/918] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:12:56 2022] GET /identity/v3/role_assignments?user.id=77fb7ab7232b48a286b7798beca19f58&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=06dfeb3edc3b4716b86a7f7be42d71b1 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:57.162373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e89cc1-f629-4be9-ab5f-cde4a4dcb38c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:57.162556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e89cc1-f629-4be9-ab5f-cde4a4dcb38c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:57.162834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e89cc1-f629-4be9-ab5f-cde4a4dcb38c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:57.164567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 462/919] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:57.170529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37ec87-5f83-408c-8ee9-9fcd582a7d86 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:57.170865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37ec87-5f83-408c-8ee9-9fcd582a7d86 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:57.171098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af37ec87-5f83-408c-8ee9-9fcd582a7d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:57.178956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:57.191245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-af37ec87-5f83-408c-8ee9-9fcd582a7d86 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:57.192474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-af37ec87-5f83-408c-8ee9-9fcd582a7d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:57.274226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 458/920] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:57.295970 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e89cc1-f629-4be9-ab5f-cde4a4dcb38c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:57.327421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:57.400255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e89cc1-f629-4be9-ab5f-cde4a4dcb38c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:57.439243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16173424-68f2-4af7-a7ff-0e7f44bdf35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:57.439828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16173424-68f2-4af7-a7ff-0e7f44bdf35e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:57.440110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16173424-68f2-4af7-a7ff-0e7f44bdf35e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:57.440705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16173424-68f2-4af7-a7ff-0e7f44bdf35e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:57.441055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16173424-68f2-4af7-a7ff-0e7f44bdf35e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:57.442514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16173424-68f2-4af7-a7ff-0e7f44bdf35e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-16173424-68f2-4af7-a7ff-0e7f44bdf35e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:57.452231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:57.453424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:57.454486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 463/921] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:12:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:57.462928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37ec87-5f83-408c-8ee9-9fcd582a7d86 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:57.567802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af37ec87-5f83-408c-8ee9-9fcd582a7d86 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:57.608086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f87a3-56d5-4eac-9c94-84ad23286b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:57.608730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f87a3-56d5-4eac-9c94-84ad23286b09 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:57.609376 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f87a3-56d5-4eac-9c94-84ad23286b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:57.609650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f87a3-56d5-4eac-9c94-84ad23286b09 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:57.610380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f87a3-56d5-4eac-9c94-84ad23286b09 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:57.610682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f87a3-56d5-4eac-9c94-84ad23286b09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:57.612121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f87a3-56d5-4eac-9c94-84ad23286b09 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:57.613539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:57.618284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 459/922] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jul 26 07:12:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:57.627462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16173424-68f2-4af7-a7ff-0e7f44bdf35e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:57.669833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cfdfb-ded4-4b8b-973f-0ea736ec2093 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:57.670204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cfdfb-ded4-4b8b-973f-0ea736ec2093 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:57.670556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cfdfb-ded4-4b8b-973f-0ea736ec2093 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:57.672255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 460/923] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:12:57.677268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee393d53-b942-4248-a61c-93c7b6c09587 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:57.677618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee393d53-b942-4248-a61c-93c7b6c09587 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:57.677989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee393d53-b942-4248-a61c-93c7b6c09587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:57.697630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ee393d53-b942-4248-a61c-93c7b6c09587 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:57.699310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ee393d53-b942-4248-a61c-93c7b6c09587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:57.734257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:57.742654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16173424-68f2-4af7-a7ff-0e7f44bdf35e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:57.782176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 461/924] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:12:57.784397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b3eee-1f44-4a83-bbe2-5ca43655fcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:57.785163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b3eee-1f44-4a83-bbe2-5ca43655fcbc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:57.785514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b3eee-1f44-4a83-bbe2-5ca43655fcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:57.785876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547b3eee-1f44-4a83-bbe2-5ca43655fcbc None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:57.786639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b3eee-1f44-4a83-bbe2-5ca43655fcbc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:57.787006 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b3eee-1f44-4a83-bbe2-5ca43655fcbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:57.788448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547b3eee-1f44-4a83-bbe2-5ca43655fcbc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:57.793922 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 464/925] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jul 26 07:12:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:57.800028 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b3eee-1f44-4a83-bbe2-5ca43655fcbc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:57.801355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee393d53-b942-4248-a61c-93c7b6c09587 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:57.903706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee393d53-b942-4248-a61c-93c7b6c09587 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:57.905070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547b3eee-1f44-4a83-bbe2-5ca43655fcbc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:57.938657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:57.941988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:57.942698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:57.943286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:57.943775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:57.944654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:57.946487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:57.948348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1253c15-e9db-4096-a875-a9b6f317afc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:57.948928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1253c15-e9db-4096-a875-a9b6f317afc4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:57.949213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1253c15-e9db-4096-a875-a9b6f317afc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:57.949486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1253c15-e9db-4096-a875-a9b6f317afc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:57.949603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WARNING keystone.server.flask.common [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 26 07:12:57.950287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1253c15-e9db-4096-a875-a9b6f317afc4 None admin] RBAC: Adding query filter params (interface=public, service_id=1d76c2fd275844d4a581c3c1d741f6a7, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:57.950695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1253c15-e9db-4096-a875-a9b6f317afc4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:57.952238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1253c15-e9db-4096-a875-a9b6f317afc4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:57.961927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 462/926] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jul 26 07:12:57 2022] GET /identity/v3/endpoints?service_id=1d76c2fd275844d4a581c3c1d741f6a7&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:57.964716 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1253c15-e9db-4096-a875-a9b6f317afc4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:57.973845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 07:12:57.973845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 07:12:57.973845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [parameters: {'id': 'd9eda7f91da24f8ba1073b310301b7ed', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 07:12:57.973845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9eda7f91da24f8ba1073b310301b7ed.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9eda7f91da24f8ba1073b310301b7ed. Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 07:12:57.991463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 26 07:12:57.992364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jul 26 07:12:57.993344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 07:12:57.994296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 07:12:57.995214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 07:12:57.996025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'id': 'd9eda7f91da24f8ba1073b310301b7ed', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:57.996891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:57.997675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d9eda7f91da24f8ba1073b310301b7ed. Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 465/927] 10.222.0.39 () {60 vars in 1140 bytes} [Tue Jul 26 07:12:57 2022] POST /identity/v3/projects => generated 262 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:58.000968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:58.018282 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:58.086518 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1253c15-e9db-4096-a875-a9b6f317afc4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:58.110814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c13d3-331f-45bc-800d-dd282f58a104 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:58.123696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e416f3-d910-4ff3-8b54-1fe471379ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:58.124721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e416f3-d910-4ff3-8b54-1fe471379ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:58.125223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e416f3-d910-4ff3-8b54-1fe471379ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:58.125694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e416f3-d910-4ff3-8b54-1fe471379ad3 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:58.126608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e416f3-d910-4ff3-8b54-1fe471379ad3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:58.128196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e416f3-d910-4ff3-8b54-1fe471379ad3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:58.142197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 463/928] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jul 26 07:12:57 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:12:58.149106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047f0fca-ecc4-4611-a827-ba3d8f09e09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:58.149833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f0fca-ecc4-4611-a827-ba3d8f09e09b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:58.150119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f0fca-ecc4-4611-a827-ba3d8f09e09b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:58.150391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047f0fca-ecc4-4611-a827-ba3d8f09e09b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:58.158933 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f0fca-ecc4-4611-a827-ba3d8f09e09b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:58.160986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047f0fca-ecc4-4611-a827-ba3d8f09e09b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-047f0fca-ecc4-4611-a827-ba3d8f09e09b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:12:58.174734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:58.175794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:58.176674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:12:58.176674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:12:58.176674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:12:58.176674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:12:58.176674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:12:58.176674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:12:58.176674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:12:58.176674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:58.176674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 466/929] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:12:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:58.182522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e416f3-d910-4ff3-8b54-1fe471379ad3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:58.249476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:58.299122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e416f3-d910-4ff3-8b54-1fe471379ad3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:58.337015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5731d5c-0327-40e9-99bf-f442fe06b5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:58.337665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5731d5c-0327-40e9-99bf-f442fe06b5ed None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:58.337969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5731d5c-0327-40e9-99bf-f442fe06b5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:58.338257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5731d5c-0327-40e9-99bf-f442fe06b5ed None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:58.338989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5731d5c-0327-40e9-99bf-f442fe06b5ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:58.339294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5731d5c-0327-40e9-99bf-f442fe06b5ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:58.340742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5731d5c-0327-40e9-99bf-f442fe06b5ed None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:58.359230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 464/930] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:12:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:58.563554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294d5c-56ab-4385-ac2b-58aa8d4d10e9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:58.564119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294d5c-56ab-4385-ac2b-58aa8d4d10e9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:58.564119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b294d5c-56ab-4385-ac2b-58aa8d4d10e9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:58.568689 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 467/931] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:58.574561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec80e5-3346-4b4a-8147-14a29818a4f4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:58.575556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec80e5-3346-4b4a-8147-14a29818a4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:58.576049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec80e5-3346-4b4a-8147-14a29818a4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:58.596330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-40ec80e5-3346-4b4a-8147-14a29818a4f4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:58.597962 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-40ec80e5-3346-4b4a-8147-14a29818a4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:58.673761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 465/932] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:58.691465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b294d5c-56ab-4385-ac2b-58aa8d4d10e9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:58.774180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:58.801584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b294d5c-56ab-4385-ac2b-58aa8d4d10e9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:58.841847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e07f3-eb4c-4aa0-8897-ad569657ff10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:58.841847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e07f3-eb4c-4aa0-8897-ad569657ff10 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:58.841847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e07f3-eb4c-4aa0-8897-ad569657ff10 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:58.841847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e07f3-eb4c-4aa0-8897-ad569657ff10 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:58.842243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e07f3-eb4c-4aa0-8897-ad569657ff10 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:58.842243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e07f3-eb4c-4aa0-8897-ad569657ff10 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5d1e07f3-eb4c-4aa0-8897-ad569657ff10 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:58.851174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:58.852110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:12:58.853403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 468/933] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Jul 26 07:12:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:58.861085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec80e5-3346-4b4a-8147-14a29818a4f4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:58.909550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:58.964328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec80e5-3346-4b4a-8147-14a29818a4f4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:59.001500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a46b3-13fe-4704-a71f-0e812e68459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:59.002647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a46b3-13fe-4704-a71f-0e812e68459d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:59.003141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a46b3-13fe-4704-a71f-0e812e68459d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:59.003720 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a46b3-13fe-4704-a71f-0e812e68459d None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:59.004605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a46b3-13fe-4704-a71f-0e812e68459d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:59.005130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a46b3-13fe-4704-a71f-0e812e68459d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:59.006827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a46b3-13fe-4704-a71f-0e812e68459d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:59.017327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 466/934] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:12:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:59.019048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e07f3-eb4c-4aa0-8897-ad569657ff10 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:59.139583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30022a25-14ae-462e-a4e5-1313312283df None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:59.139759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e07f3-eb4c-4aa0-8897-ad569657ff10 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:59.139839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30022a25-14ae-462e-a4e5-1313312283df None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:59.140083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30022a25-14ae-462e-a4e5-1313312283df None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:59.141990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 467/935] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:12:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:12:59.147875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c58e90-423c-4e78-86e1-9216506a8e30 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:59.148155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c58e90-423c-4e78-86e1-9216506a8e30 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:59.148433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c58e90-423c-4e78-86e1-9216506a8e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:59.166852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-32c58e90-423c-4e78-86e1-9216506a8e30 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:12:59.168054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-32c58e90-423c-4e78-86e1-9216506a8e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:12:59.176486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a2a26-a7f5-4148-86f9-fa94fccdb8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:59.177124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a2a26-a7f5-4148-86f9-fa94fccdb8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:59.177408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a2a26-a7f5-4148-86f9-fa94fccdb8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:59.177686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a2a26-a7f5-4148-86f9-fa94fccdb8d2 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:59.178406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a2a26-a7f5-4148-86f9-fa94fccdb8d2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:59.178725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a2a26-a7f5-4148-86f9-fa94fccdb8d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:59.180103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a2a26-a7f5-4148-86f9-fa94fccdb8d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:59.185793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 469/936] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:12:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:59.194843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a2a26-a7f5-4148-86f9-fa94fccdb8d2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:59.199018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:59.253780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 468/937] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:12:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:12:59.275189 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c58e90-423c-4e78-86e1-9216506a8e30 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:59.319990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a2a26-a7f5-4148-86f9-fa94fccdb8d2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:59.323760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:59.367264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ff3bc-c070-4eb7-a8ec-de43ea8c7693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:59.368290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ff3bc-c070-4eb7-a8ec-de43ea8c7693 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:59.368790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ff3bc-c070-4eb7-a8ec-de43ea8c7693 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:59.369259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031ff3bc-c070-4eb7-a8ec-de43ea8c7693 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:59.370351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ff3bc-c070-4eb7-a8ec-de43ea8c7693 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:59.372146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031ff3bc-c070-4eb7-a8ec-de43ea8c7693 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:59.396618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 470/938] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:12:59 2022] POST /identity/v3/endpoints => generated 347 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:12:59.398847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c58e90-423c-4e78-86e1-9216506a8e30 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:59.458279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0ff4a-c581-458f-b317-653874b7930d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:59.458955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0ff4a-c581-458f-b317-653874b7930d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:59.459230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0ff4a-c581-458f-b317-653874b7930d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:59.459496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab0ff4a-c581-458f-b317-653874b7930d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:59.473060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab0ff4a-c581-458f-b317-653874b7930d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-aab0ff4a-c581-458f-b317-653874b7930d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:12:59.478434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:59.479413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:59.480270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 469/939] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jul 26 07:12:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:12:59.487122 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ff3bc-c070-4eb7-a8ec-de43ea8c7693 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:59.552766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:59.602726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031ff3bc-c070-4eb7-a8ec-de43ea8c7693 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:59.641654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0b3b7-6e9a-4712-818a-71b27bda77e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:59.642369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0b3b7-6e9a-4712-818a-71b27bda77e6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:59.642656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0b3b7-6e9a-4712-818a-71b27bda77e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:59.642934 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0b3b7-6e9a-4712-818a-71b27bda77e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:59.643664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0b3b7-6e9a-4712-818a-71b27bda77e6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:12:59.643972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0b3b7-6e9a-4712-818a-71b27bda77e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:59.645452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0b3b7-6e9a-4712-818a-71b27bda77e6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:59.655522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 471/940] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jul 26 07:12:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:12:59.661841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0ff4a-c581-458f-b317-653874b7930d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:59.768256 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:12:59.776382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab0ff4a-c581-458f-b317-653874b7930d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:59.814491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead94d74-277f-4f60-8b1c-de1f973841ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:12:59.815239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead94d74-277f-4f60-8b1c-de1f973841ab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:12:59.815507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead94d74-277f-4f60-8b1c-de1f973841ab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:12:59.815873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead94d74-277f-4f60-8b1c-de1f973841ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:12:59.827656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead94d74-277f-4f60-8b1c-de1f973841ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:12:59.829264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead94d74-277f-4f60-8b1c-de1f973841ab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ead94d74-277f-4f60-8b1c-de1f973841ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:12:59.845504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:12:59.846493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:12:59.847971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:12:59.847971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:12:59.847971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:12:59.847971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:12:59.847971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 470/941] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:12:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:12:59.853534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0b3b7-6e9a-4712-818a-71b27bda77e6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:12:59.972514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0b3b7-6e9a-4712-818a-71b27bda77e6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:12:59.995437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:00.013594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c98b1a-6792-4c51-99e6-690ab1a091ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:00.014271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c98b1a-6792-4c51-99e6-690ab1a091ee None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.014556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c98b1a-6792-4c51-99e6-690ab1a091ee None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.014833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c98b1a-6792-4c51-99e6-690ab1a091ee None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.015572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c98b1a-6792-4c51-99e6-690ab1a091ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:00.015981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c98b1a-6792-4c51-99e6-690ab1a091ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:00.017487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c98b1a-6792-4c51-99e6-690ab1a091ee None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:00.032541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 472/942] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:00.038356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead94d74-277f-4f60-8b1c-de1f973841ab None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:00.154245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead94d74-277f-4f60-8b1c-de1f973841ab None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:00.194067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac03d2-860a-404e-aaec-c67a157f6c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:00.195123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac03d2-860a-404e-aaec-c67a157f6c39 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.195618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac03d2-860a-404e-aaec-c67a157f6c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.196107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac03d2-860a-404e-aaec-c67a157f6c39 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.205410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac03d2-860a-404e-aaec-c67a157f6c39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:00.207450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac03d2-860a-404e-aaec-c67a157f6c39 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d5ac03d2-860a-404e-aaec-c67a157f6c39 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:13:00.220835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:00.221871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:00.222914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:00.222914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:00.222914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:13:00.222914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:13:00.222914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:13:00.222914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:13:00.222914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:13:00.222914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:13:00.222914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:13:00.222914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:00.225688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 471/943] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jul 26 07:13:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:00.231879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c98b1a-6792-4c51-99e6-690ab1a091ee None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:00.273107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac7bcc-95be-41c3-8c91-d6fca4735f23 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.273433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac7bcc-95be-41c3-8c91-d6fca4735f23 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.273662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ac7bcc-95be-41c3-8c91-d6fca4735f23 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.279653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 472/944] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:00.281155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65733f58-d810-44ef-bc57-07ad25e72b1a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.281455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65733f58-d810-44ef-bc57-07ad25e72b1a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.281755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65733f58-d810-44ef-bc57-07ad25e72b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.302978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-65733f58-d810-44ef-bc57-07ad25e72b1a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:00.304487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-65733f58-d810-44ef-bc57-07ad25e72b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:00.306920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:00.348716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c98b1a-6792-4c51-99e6-690ab1a091ee None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:00.387270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d6654-738a-4a21-b719-1bc042825179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:00.387928 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d6654-738a-4a21-b719-1bc042825179 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.388201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d6654-738a-4a21-b719-1bc042825179 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.388475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d6654-738a-4a21-b719-1bc042825179 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.389283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d6654-738a-4a21-b719-1bc042825179 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:00.389592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d6654-738a-4a21-b719-1bc042825179 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:00.391056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d6654-738a-4a21-b719-1bc042825179 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:00.405846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 473/945] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:00.415754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 473/946] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jul 26 07:13:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:00.423016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65733f58-d810-44ef-bc57-07ad25e72b1a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:00.424275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d6654-738a-4a21-b719-1bc042825179 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:00.481970 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:00.540564 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65733f58-d810-44ef-bc57-07ad25e72b1a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:00.568353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d6654-738a-4a21-b719-1bc042825179 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:00.577215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d4afa-446e-4dc4-be67-0bc838369785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:00.577994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d4afa-446e-4dc4-be67-0bc838369785 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.578335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d4afa-446e-4dc4-be67-0bc838369785 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.578672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d4afa-446e-4dc4-be67-0bc838369785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.586358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:00.590561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d4afa-446e-4dc4-be67-0bc838369785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:00.592477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d4afa-446e-4dc4-be67-0bc838369785 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-9c6d4afa-446e-4dc4-be67-0bc838369785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:00.612470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:00.613513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:00.614471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:13:00.614471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:00.614471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:00.614471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:00.614471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 474/947] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:00.619639 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d4afa-446e-4dc4-be67-0bc838369785 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:00.624557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab95655-b451-43fe-9581-d95bfe304b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:00.625940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab95655-b451-43fe-9581-d95bfe304b1a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.626478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab95655-b451-43fe-9581-d95bfe304b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.626951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab95655-b451-43fe-9581-d95bfe304b1a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.627804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab95655-b451-43fe-9581-d95bfe304b1a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:00.630043 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab95655-b451-43fe-9581-d95bfe304b1a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bab95655-b451-43fe-9581-d95bfe304b1a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:00.643675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:00.644814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:13:00.645884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:00.653183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 474/948] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:13:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:00.653183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1881230-8c1a-4ad7-a3ae-b4dce29155a1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.653183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1881230-8c1a-4ad7-a3ae-b4dce29155a1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.653183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1881230-8c1a-4ad7-a3ae-b4dce29155a1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.657208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 475/949] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:00.658058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1881230-8c1a-4ad7-a3ae-b4dce29155a1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:00.739457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d4afa-446e-4dc4-be67-0bc838369785 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:00.775281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3e63c-897a-4293-9f82-f73e06c5bb3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:00.775918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3e63c-897a-4293-9f82-f73e06c5bb3b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.776196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3e63c-897a-4293-9f82-f73e06c5bb3b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.776468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3e63c-897a-4293-9f82-f73e06c5bb3b None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.777160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3e63c-897a-4293-9f82-f73e06c5bb3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:00.777463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3e63c-897a-4293-9f82-f73e06c5bb3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:00.778594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1881230-8c1a-4ad7-a3ae-b4dce29155a1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:00.778871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3e63c-897a-4293-9f82-f73e06c5bb3b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:00.780407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:00.791909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 475/950] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:00.793173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4a665-2c3e-481f-91f3-a9b6a42c9ff2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.793463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4a665-2c3e-481f-91f3-a9b6a42c9ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.793818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4a665-2c3e-481f-91f3-a9b6a42c9ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.811541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-19a4a665-2c3e-481f-91f3-a9b6a42c9ff2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:00.812734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19a4a665-2c3e-481f-91f3-a9b6a42c9ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:00.820254 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2aa48-0ff8-48a6-9a96-8aac7c9e0445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:00.821009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2aa48-0ff8-48a6-9a96-8aac7c9e0445 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.821356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2aa48-0ff8-48a6-9a96-8aac7c9e0445 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.821698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2aa48-0ff8-48a6-9a96-8aac7c9e0445 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.822431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2aa48-0ff8-48a6-9a96-8aac7c9e0445 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:00.822722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2aa48-0ff8-48a6-9a96-8aac7c9e0445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:00.824123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2aa48-0ff8-48a6-9a96-8aac7c9e0445 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:00.829065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 476/951] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jul 26 07:13:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:00.829871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2aa48-0ff8-48a6-9a96-8aac7c9e0445 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:00.867560 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:00.898196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 476/952] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:00.899144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4a665-2c3e-481f-91f3-a9b6a42c9ff2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:00.937891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2aa48-0ff8-48a6-9a96-8aac7c9e0445 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:00.976043 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a9836-f384-4b61-aa41-9899772ab30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:00.976697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a9836-f384-4b61-aa41-9899772ab30b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:00.976977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a9836-f384-4b61-aa41-9899772ab30b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:00.977585 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921a9836-f384-4b61-aa41-9899772ab30b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:00.984950 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a9836-f384-4b61-aa41-9899772ab30b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:00.986150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921a9836-f384-4b61-aa41-9899772ab30b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-921a9836-f384-4b61-aa41-9899772ab30b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:13:00.997640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:00.998557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:00.999583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:13:00.999583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:13:00.999583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:13:00.999583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:13:00.999583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:13:00.999583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:13:00.999583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:13:00.999583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:00.999583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 477/953] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:13:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:00.999583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a9836-f384-4b61-aa41-9899772ab30b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:01.017301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4a665-2c3e-481f-91f3-a9b6a42c9ff2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:01.038429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:01.061123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9de47-0d23-482a-9bbd-5c75d8352196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:01.061772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9de47-0d23-482a-9bbd-5c75d8352196 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:01.062052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9de47-0d23-482a-9bbd-5c75d8352196 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:01.062345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9de47-0d23-482a-9bbd-5c75d8352196 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:01.063030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9de47-0d23-482a-9bbd-5c75d8352196 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:01.063326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9de47-0d23-482a-9bbd-5c75d8352196 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:01.064712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9de47-0d23-482a-9bbd-5c75d8352196 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:01.071326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 477/954] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jul 26 07:13:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:01.072266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9de47-0d23-482a-9bbd-5c75d8352196 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:01.126348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:01.127434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921a9836-f384-4b61-aa41-9899772ab30b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:01.165071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41f4e9-f561-4694-b63d-0612b1d5017d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:01.165698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41f4e9-f561-4694-b63d-0612b1d5017d None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:01.166300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41f4e9-f561-4694-b63d-0612b1d5017d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:01.166571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41f4e9-f561-4694-b63d-0612b1d5017d None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:01.167404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41f4e9-f561-4694-b63d-0612b1d5017d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:01.168836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41f4e9-f561-4694-b63d-0612b1d5017d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:01.185013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.sql.core [None req-4a41f4e9-f561-4694-b63d-0612b1d5017d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 07:13:01.185013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 07:13:01.185013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [parameters: {'user_id': 'ac3e0e213a60421fa7f30084d0cbfe7d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 07:13:01.185013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80170) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 26 07:13:01.188815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9de47-0d23-482a-9bbd-5c75d8352196 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-4a41f4e9-f561-4694-b63d-0612b1d5017d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 26 07:13:01.192763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 07:13:01.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application yield resource Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application next(self.gen) Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.rollback() Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.force_reraise() Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise self.value Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application session.commit() Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 26 07:13:01.194786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._prepare_impl() Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.session.flush() Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._flush(objects) Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application compat.raise_( Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application flush_context.execute() Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rec.execute(self) Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 26 07:13:01.195700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 26 07:13:01.196651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result = self._query(query) Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application conn.query(q) Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application result.read() Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 26 07:13:01.197517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ac3e0e213a60421fa7f30084d0cbfe7d', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:01.201287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 26 07:13:01.203750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 26 07:13:01.206208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 26 07:13:01.206208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 26 07:13:01.206208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 26 07:13:01.206208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 26 07:13:01.206208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 26 07:13:01.206208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:01.206208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 478/955] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Jul 26 07:13:00 2022] POST /identity/v3/users => generated 160 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:01.206208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41f4e9-f561-4694-b63d-0612b1d5017d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:01.233430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae5356-77cd-47ea-9c5c-4998d6ec1e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:01.234513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae5356-77cd-47ea-9c5c-4998d6ec1e61 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:01.234800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae5356-77cd-47ea-9c5c-4998d6ec1e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:01.235251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae5356-77cd-47ea-9c5c-4998d6ec1e61 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:01.236826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae5356-77cd-47ea-9c5c-4998d6ec1e61 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:01.237147 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae5356-77cd-47ea-9c5c-4998d6ec1e61 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:01.238672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ae5356-77cd-47ea-9c5c-4998d6ec1e61 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:01.257809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 478/956] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:13:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:01.257809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae5356-77cd-47ea-9c5c-4998d6ec1e61 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:01.294450 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:01.310290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41f4e9-f561-4694-b63d-0612b1d5017d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:01.353927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce0ac1-a54f-4aae-8033-4815a706209a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:01.354746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce0ac1-a54f-4aae-8033-4815a706209a None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:01.355125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce0ac1-a54f-4aae-8033-4815a706209a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:01.355494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce0ac1-a54f-4aae-8033-4815a706209a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:01.356316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce0ac1-a54f-4aae-8033-4815a706209a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:01.358081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce0ac1-a54f-4aae-8033-4815a706209a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:01.377866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae5356-77cd-47ea-9c5c-4998d6ec1e61 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:01.381057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:01.394487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 479/957] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:13:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:13:01.395174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce0ac1-a54f-4aae-8033-4815a706209a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:01.465332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea9acf-0688-42be-afa4-a36b93b000b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:01.466438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea9acf-0688-42be-afa4-a36b93b000b9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:01.466438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea9acf-0688-42be-afa4-a36b93b000b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:01.466611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea9acf-0688-42be-afa4-a36b93b000b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:01.475808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea9acf-0688-42be-afa4-a36b93b000b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:01.477432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea9acf-0688-42be-afa4-a36b93b000b9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-beea9acf-0688-42be-afa4-a36b93b000b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:13:01.490933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:01.491960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:01.492936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:01.492936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:01.492936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:13:01.492936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:13:01.492936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:13:01.492936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:13:01.492936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:13:01.492936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:13:01.492936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 26 07:13:01.492936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:01.497317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 479/958] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:13:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:01.498799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea9acf-0688-42be-afa4-a36b93b000b9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:01.543643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce0ac1-a54f-4aae-8033-4815a706209a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:01.585043 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91b230a-01de-410a-9fba-64c28dc9509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:01.585794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b230a-01de-410a-9fba-64c28dc9509e None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:01.586371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b230a-01de-410a-9fba-64c28dc9509e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:01.586655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91b230a-01de-410a-9fba-64c28dc9509e None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/77fb7ab7232b48a286b7798beca19f58/roles/06dfeb3edc3b4716b86a7f7be42d71b1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:01.589564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b230a-01de-410a-9fba-64c28dc9509e None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=77fb7ab7232b48a286b7798beca19f58, role_id=06dfeb3edc3b4716b86a7f7be42d71b1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:01.591095 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91b230a-01de-410a-9fba-64c28dc9509e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:01.606590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 480/959] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:13:01 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/77fb7ab7232b48a286b7798beca19f58/roles/06dfeb3edc3b4716b86a7f7be42d71b1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:13:01.607965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c266b60-75fe-4828-b49f-3ede775768b6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:01.608259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c266b60-75fe-4828-b49f-3ede775768b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:01.608541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c266b60-75fe-4828-b49f-3ede775768b6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:01.610233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 481/960] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:01.612346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea9acf-0688-42be-afa4-a36b93b000b9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:01.617189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cd2d8-433a-4652-be8f-bcb766140b50 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:01.617534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cd2d8-433a-4652-be8f-bcb766140b50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:01.617793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cd2d8-433a-4652-be8f-bcb766140b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:01.642896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:01.644557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f43cd2d8-433a-4652-be8f-bcb766140b50 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:01.648667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f43cd2d8-433a-4652-be8f-bcb766140b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:01.664766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c79a74-8194-4f6b-98fd-53f72b1cedeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:01.665434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c79a74-8194-4f6b-98fd-53f72b1cedeb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:01.665726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c79a74-8194-4f6b-98fd-53f72b1cedeb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:01.666075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c79a74-8194-4f6b-98fd-53f72b1cedeb None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:01.666876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c79a74-8194-4f6b-98fd-53f72b1cedeb None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:01.667186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c79a74-8194-4f6b-98fd-53f72b1cedeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:01.669562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c79a74-8194-4f6b-98fd-53f72b1cedeb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:01.720652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 480/961] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Jul 26 07:13:01 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:01.737672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 482/962] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:01.760436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c79a74-8194-4f6b-98fd-53f72b1cedeb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:01.784394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:01.884918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c79a74-8194-4f6b-98fd-53f72b1cedeb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:01.924434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44665e47-c3bf-41fe-abcb-6c635a82ecc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:01.925098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44665e47-c3bf-41fe-abcb-6c635a82ecc2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:01.925383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44665e47-c3bf-41fe-abcb-6c635a82ecc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:01.925698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44665e47-c3bf-41fe-abcb-6c635a82ecc2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:01.926400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44665e47-c3bf-41fe-abcb-6c635a82ecc2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:01.927782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44665e47-c3bf-41fe-abcb-6c635a82ecc2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-44665e47-c3bf-41fe-abcb-6c635a82ecc2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:01.938567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:01.939682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:01.940526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 481/963] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:13:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:01.945891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cd2d8-433a-4652-be8f-bcb766140b50 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:02.058670 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cd2d8-433a-4652-be8f-bcb766140b50 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:02.094929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18d503-0829-4a66-b375-33c431d2ce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:02.095559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18d503-0829-4a66-b375-33c431d2ce4f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:02.095838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18d503-0829-4a66-b375-33c431d2ce4f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:02.096117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18d503-0829-4a66-b375-33c431d2ce4f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:02.096808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18d503-0829-4a66-b375-33c431d2ce4f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:02.097107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18d503-0829-4a66-b375-33c431d2ce4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:02.098584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18d503-0829-4a66-b375-33c431d2ce4f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:02.099826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:02.105901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 483/964] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:13:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:02.111440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44665e47-c3bf-41fe-abcb-6c635a82ecc2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:02.203406 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:02.223930 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44665e47-c3bf-41fe-abcb-6c635a82ecc2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:02.259078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e956179-3f2a-422c-8442-de93c937a865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:02.259707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e956179-3f2a-422c-8442-de93c937a865 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:02.259983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e956179-3f2a-422c-8442-de93c937a865 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:02.260249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e956179-3f2a-422c-8442-de93c937a865 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:02.260917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e956179-3f2a-422c-8442-de93c937a865 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:02.261217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e956179-3f2a-422c-8442-de93c937a865 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:02.262616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e956179-3f2a-422c-8442-de93c937a865 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:02.267427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 482/965] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:13:02 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:03.428664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13af341-5aae-41b3-9e70-d226e816b31c None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:03.429205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13af341-5aae-41b3-9e70-d226e816b31c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:03.429205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13af341-5aae-41b3-9e70-d226e816b31c None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:03.430910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 484/966] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:03.438603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830d59f-3cb5-41be-ab2b-bcef5a6e5a9e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:03.438603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830d59f-3cb5-41be-ab2b-bcef5a6e5a9e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:03.438603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0830d59f-3cb5-41be-ab2b-bcef5a6e5a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:03.456781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0830d59f-3cb5-41be-ab2b-bcef5a6e5a9e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:03.458037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0830d59f-3cb5-41be-ab2b-bcef5a6e5a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:03.535934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 483/967] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:03.558080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13af341-5aae-41b3-9e70-d226e816b31c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:03.663222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13af341-5aae-41b3-9e70-d226e816b31c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:03.683582 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ebc7d-550a-4c23-9c65-a26764fd3654 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:03.683771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ebc7d-550a-4c23-9c65-a26764fd3654 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:03.684013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ebc7d-550a-4c23-9c65-a26764fd3654 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:03.685594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 484/968] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:13:03.699640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edab096-1b0d-4f45-8f28-6e6aa5cf6094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:03.700299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edab096-1b0d-4f45-8f28-6e6aa5cf6094 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:03.700582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edab096-1b0d-4f45-8f28-6e6aa5cf6094 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:03.700854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edab096-1b0d-4f45-8f28-6e6aa5cf6094 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:03.700950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9e80b-273d-486d-9b95-1c59c3a9c524 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:03.701232 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9e80b-273d-486d-9b95-1c59c3a9c524 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:03.701488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edab096-1b0d-4f45-8f28-6e6aa5cf6094 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:03.701581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9e80b-273d-486d-9b95-1c59c3a9c524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:03.702876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edab096-1b0d-4f45-8f28-6e6aa5cf6094 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-9edab096-1b0d-4f45-8f28-6e6aa5cf6094 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:03.719103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:03.720326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:13:03.721198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:03.723733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bda9e80b-273d-486d-9b95-1c59c3a9c524 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:03.725061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bda9e80b-273d-486d-9b95-1c59c3a9c524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:03.726093 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 485/969] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:13:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:03.727399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edab096-1b0d-4f45-8f28-6e6aa5cf6094 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:03.801318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:03.801913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 485/970] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:13:03.824907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9e80b-273d-486d-9b95-1c59c3a9c524 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:03.833945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:03.848822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edab096-1b0d-4f45-8f28-6e6aa5cf6094 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:03.887095 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e43a2-2e5c-4dde-854b-7666156ec6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:03.887726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e43a2-2e5c-4dde-854b-7666156ec6ec None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:03.888002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e43a2-2e5c-4dde-854b-7666156ec6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:03.888269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e43a2-2e5c-4dde-854b-7666156ec6ec None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:03.888945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e43a2-2e5c-4dde-854b-7666156ec6ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:03.889241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e43a2-2e5c-4dde-854b-7666156ec6ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:03.890613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e43a2-2e5c-4dde-854b-7666156ec6ec None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:03.895708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 486/971] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jul 26 07:13:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:03.901258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e43a2-2e5c-4dde-854b-7666156ec6ec None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:03.936139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9e80b-273d-486d-9b95-1c59c3a9c524 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:03.971006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02972609-b95c-4f95-a943-0694e4b165f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:03.971665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02972609-b95c-4f95-a943-0694e4b165f9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:03.971946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02972609-b95c-4f95-a943-0694e4b165f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:03.972218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02972609-b95c-4f95-a943-0694e4b165f9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:03.982477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02972609-b95c-4f95-a943-0694e4b165f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-02972609-b95c-4f95-a943-0694e4b165f9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:13:03.987810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:03.989006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:03.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 486/972] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jul 26 07:13:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:03.995093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02972609-b95c-4f95-a943-0694e4b165f9 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:04.018275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e43a2-2e5c-4dde-854b-7666156ec6ec None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:04.046918 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:04.058545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d14e3-9233-4a9b-a221-f0e080c0c03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:04.059591 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d14e3-9233-4a9b-a221-f0e080c0c03d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.060062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d14e3-9233-4a9b-a221-f0e080c0c03d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.060536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4d14e3-9233-4a9b-a221-f0e080c0c03d None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.061782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d14e3-9233-4a9b-a221-f0e080c0c03d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:04.062348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d14e3-9233-4a9b-a221-f0e080c0c03d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:04.064817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4d14e3-9233-4a9b-a221-f0e080c0c03d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:04.071252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 487/973] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jul 26 07:13:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:04.077112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d14e3-9233-4a9b-a221-f0e080c0c03d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:04.090678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:04.132602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02972609-b95c-4f95-a943-0694e4b165f9 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:04.171487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2029a12-481b-4277-931c-9fe2d1b46cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:04.172119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2029a12-481b-4277-931c-9fe2d1b46cef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.172402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2029a12-481b-4277-931c-9fe2d1b46cef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.172678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2029a12-481b-4277-931c-9fe2d1b46cef None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.173399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2029a12-481b-4277-931c-9fe2d1b46cef None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:04.173702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2029a12-481b-4277-931c-9fe2d1b46cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:04.175105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2029a12-481b-4277-931c-9fe2d1b46cef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:04.189216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 487/974] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jul 26 07:13:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:04.190407 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2029a12-481b-4277-931c-9fe2d1b46cef None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:04.207287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4d14e3-9233-4a9b-a221-f0e080c0c03d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:04.242248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c30e5b-6d99-4d9c-afe9-5268b0d493fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:04.242870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c30e5b-6d99-4d9c-afe9-5268b0d493fc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.243159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c30e5b-6d99-4d9c-afe9-5268b0d493fc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.243415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c30e5b-6d99-4d9c-afe9-5268b0d493fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.244261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c30e5b-6d99-4d9c-afe9-5268b0d493fc None admin] RBAC: Adding query filter params (interface=admin, service_id=1d76c2fd275844d4a581c3c1d741f6a7, region_id=RegionOne) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:04.244569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c30e5b-6d99-4d9c-afe9-5268b0d493fc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:04.246015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c30e5b-6d99-4d9c-afe9-5268b0d493fc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:04.251561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 488/975] 10.222.0.39 () {56 vars in 1249 bytes} [Tue Jul 26 07:13:04 2022] GET /identity/v3/endpoints?service_id=1d76c2fd275844d4a581c3c1d741f6a7&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:04.253141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6303c0a-e02f-4d27-a462-3cc6b6bd5ca5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.253696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6303c0a-e02f-4d27-a462-3cc6b6bd5ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.254153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6303c0a-e02f-4d27-a462-3cc6b6bd5ca5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.255768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 489/976] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:04.257069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6303c0a-e02f-4d27-a462-3cc6b6bd5ca5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:04.293798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2029a12-481b-4277-931c-9fe2d1b46cef None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:04.310599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:04.318872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:04.327338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6deca5-9ba5-46f6-8909-ecb187af8fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:04.327982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6deca5-9ba5-46f6-8909-ecb187af8fbf None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.328261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6deca5-9ba5-46f6-8909-ecb187af8fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.328529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6deca5-9ba5-46f6-8909-ecb187af8fbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.339655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6deca5-9ba5-46f6-8909-ecb187af8fbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:04.341107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6deca5-9ba5-46f6-8909-ecb187af8fbf None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ae6deca5-9ba5-46f6-8909-ecb187af8fbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:04.355561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:04.356618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:04.357521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:13:04.357521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:04.357521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:04.357521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:04.357823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6303c0a-e02f-4d27-a462-3cc6b6bd5ca5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:04.357904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 488/977] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:04.357904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881dc72-4506-44b4-b40c-7ef0e7726bae None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.358193 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881dc72-4506-44b4-b40c-7ef0e7726bae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.358576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f881dc72-4506-44b4-b40c-7ef0e7726bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.377015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f881dc72-4506-44b4-b40c-7ef0e7726bae None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:04.378229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f881dc72-4506-44b4-b40c-7ef0e7726bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:04.391649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e8f97-9902-44e0-9fc6-e4f51056c330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:04.392303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e8f97-9902-44e0-9fc6-e4f51056c330 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.392581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e8f97-9902-44e0-9fc6-e4f51056c330 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.392854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349e8f97-9902-44e0-9fc6-e4f51056c330 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.393513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e8f97-9902-44e0-9fc6-e4f51056c330 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:04.394917 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349e8f97-9902-44e0-9fc6-e4f51056c330 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:04.401231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 490/978] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jul 26 07:13:04 2022] GET /identity/v3/services => generated 1497 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:13:04.402216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e8f97-9902-44e0-9fc6-e4f51056c330 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:04.474734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 489/979] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:04.497336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881dc72-4506-44b4-b40c-7ef0e7726bae None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:04.527630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349e8f97-9902-44e0-9fc6-e4f51056c330 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:04.545936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:04.566538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bce20-f888-45b2-b916-24a55bd579b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:04.567466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bce20-f888-45b2-b916-24a55bd579b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.567466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bce20-f888-45b2-b916-24a55bd579b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.567714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6bce20-f888-45b2-b916-24a55bd579b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.568426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bce20-f888-45b2-b916-24a55bd579b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:04.568734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bce20-f888-45b2-b916-24a55bd579b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:04.570130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6bce20-f888-45b2-b916-24a55bd579b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:04.583882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 491/980] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:04.589559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bce20-f888-45b2-b916-24a55bd579b6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:04.626099 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f881dc72-4506-44b4-b40c-7ef0e7726bae None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:04.638205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:04.662782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d1afc-8d87-4f79-84ba-a843f32e4107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:04.663415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d1afc-8d87-4f79-84ba-a843f32e4107 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.663696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d1afc-8d87-4f79-84ba-a843f32e4107 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.663966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2d1afc-8d87-4f79-84ba-a843f32e4107 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.664681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d1afc-8d87-4f79-84ba-a843f32e4107 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:04.666085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2d1afc-8d87-4f79-84ba-a843f32e4107 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:04.682599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 490/981] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:13:04 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:13:04.700064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6bce20-f888-45b2-b916-24a55bd579b6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:04.748622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685221a4-42a6-4321-bf9c-e9a6d6ed4f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:04.749356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685221a4-42a6-4321-bf9c-e9a6d6ed4f4d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.749632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685221a4-42a6-4321-bf9c-e9a6d6ed4f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.749934 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685221a4-42a6-4321-bf9c-e9a6d6ed4f4d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.758452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685221a4-42a6-4321-bf9c-e9a6d6ed4f4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:04.759931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685221a4-42a6-4321-bf9c-e9a6d6ed4f4d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-685221a4-42a6-4321-bf9c-e9a6d6ed4f4d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:13:04.773286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:04.774595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:04.775537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 492/982] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jul 26 07:13:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:04.781784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d1afc-8d87-4f79-84ba-a843f32e4107 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:04.890290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2d1afc-8d87-4f79-84ba-a843f32e4107 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:04.925289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdc965-dad8-4a4d-824e-fe090bb5680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:04.926042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdc965-dad8-4a4d-824e-fe090bb5680a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:04.926325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdc965-dad8-4a4d-824e-fe090bb5680a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:04.926596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdc965-dad8-4a4d-824e-fe090bb5680a None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:04.927412 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cdc965-dad8-4a4d-824e-fe090bb5680a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:04.927720 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cdc965-dad8-4a4d-824e-fe090bb5680a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:04.929172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cdc965-dad8-4a4d-824e-fe090bb5680a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:04.952175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 491/983] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Jul 26 07:13:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:04.953847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685221a4-42a6-4321-bf9c-e9a6d6ed4f4d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:04.990289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:05.059314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685221a4-42a6-4321-bf9c-e9a6d6ed4f4d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:05.093342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d8fb8-2c74-4192-9adf-4f13fa4e9631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:05.094003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d8fb8-2c74-4192-9adf-4f13fa4e9631 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:05.094279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d8fb8-2c74-4192-9adf-4f13fa4e9631 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:05.094549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d8fb8-2c74-4192-9adf-4f13fa4e9631 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:05.163014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d8fb8-2c74-4192-9adf-4f13fa4e9631 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:05.164457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d8fb8-2c74-4192-9adf-4f13fa4e9631 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:05.197347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:05.269651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 493/984] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:04 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:13:05.275510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdc965-dad8-4a4d-824e-fe090bb5680a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:05.304666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:05.380062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdc965-dad8-4a4d-824e-fe090bb5680a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:05.415024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8a9b1-00ec-40a8-af95-9266a7daf963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:05.415841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8a9b1-00ec-40a8-af95-9266a7daf963 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:05.416215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8a9b1-00ec-40a8-af95-9266a7daf963 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:05.416584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8a9b1-00ec-40a8-af95-9266a7daf963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:05.428095 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8a9b1-00ec-40a8-af95-9266a7daf963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:05.429875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e8a9b1-00ec-40a8-af95-9266a7daf963 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b9e8a9b1-00ec-40a8-af95-9266a7daf963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:05.447785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:05.448794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:05.449698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:13:05.449698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:05.449698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:05.449698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:05.453496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 492/985] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:05.455909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d8fb8-2c74-4192-9adf-4f13fa4e9631 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:05.575879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d8fb8-2c74-4192-9adf-4f13fa4e9631 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:05.618079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3ed5a-152c-43f9-b374-4b5ae35588fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:05.618845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ed5a-152c-43f9-b374-4b5ae35588fc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:05.619312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ed5a-152c-43f9-b374-4b5ae35588fc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:05.619690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3ed5a-152c-43f9-b374-4b5ae35588fc None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:05.620667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3ed5a-152c-43f9-b374-4b5ae35588fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:05.620986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3ed5a-152c-43f9-b374-4b5ae35588fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:05.622801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3ed5a-152c-43f9-b374-4b5ae35588fc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:05.624627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:05.637341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 494/986] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:05.642846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8a9b1-00ec-40a8-af95-9266a7daf963 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:05.738682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:05.765879 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8a9b1-00ec-40a8-af95-9266a7daf963 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:05.801529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fd6e3-69f3-43cf-9e72-59a2604ba56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:05.802244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fd6e3-69f3-43cf-9e72-59a2604ba56c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:05.802525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fd6e3-69f3-43cf-9e72-59a2604ba56c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:05.802795 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51fd6e3-69f3-43cf-9e72-59a2604ba56c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:05.810579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fd6e3-69f3-43cf-9e72-59a2604ba56c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:05.812101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51fd6e3-69f3-43cf-9e72-59a2604ba56c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c51fd6e3-69f3-43cf-9e72-59a2604ba56c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:13:05.825428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:05.826491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:05.827468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:13:05.827468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:13:05.827468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:13:05.827468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:13:05.827468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:13:05.827468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:13:05.827468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:13:05.827468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:05.830165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 493/987] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:13:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:05.832641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3ed5a-152c-43f9-b374-4b5ae35588fc None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:05.939310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3ed5a-152c-43f9-b374-4b5ae35588fc None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:05.973221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18313ffe-0c20-4d0d-bd00-a9bc44aa015d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:05.973880 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313ffe-0c20-4d0d-bd00-a9bc44aa015d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:05.974160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313ffe-0c20-4d0d-bd00-a9bc44aa015d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:05.974429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18313ffe-0c20-4d0d-bd00-a9bc44aa015d None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:05.975183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18313ffe-0c20-4d0d-bd00-a9bc44aa015d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:05.975485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18313ffe-0c20-4d0d-bd00-a9bc44aa015d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:05.976981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18313ffe-0c20-4d0d-bd00-a9bc44aa015d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:05.990852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 495/988] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:13:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:05.996957 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fd6e3-69f3-43cf-9e72-59a2604ba56c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:06.105615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51fd6e3-69f3-43cf-9e72-59a2604ba56c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:06.140733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9f9b53-24b8-4bb0-b84c-cd21616461e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:06.141366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f9b53-24b8-4bb0-b84c-cd21616461e9 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:06.141645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f9b53-24b8-4bb0-b84c-cd21616461e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:06.141944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9f9b53-24b8-4bb0-b84c-cd21616461e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:06.142751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f9b53-24b8-4bb0-b84c-cd21616461e9 None admin] RBAC: Adding query filter params (role.id=06dfeb3edc3b4716b86a7f7be42d71b1, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=77fb7ab7232b48a286b7798beca19f58) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:06.143051 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f9b53-24b8-4bb0-b84c-cd21616461e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:06.144655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9f9b53-24b8-4bb0-b84c-cd21616461e9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:06.151070 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 494/989] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:13:05 2022] GET /identity/v3/role_assignments?user.id=77fb7ab7232b48a286b7798beca19f58&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=06dfeb3edc3b4716b86a7f7be42d71b1 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:06.429765 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890b266-9cba-4198-83a1-1d9901d59420 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:06.429954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890b266-9cba-4198-83a1-1d9901d59420 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:06.430229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0890b266-9cba-4198-83a1-1d9901d59420 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:06.431840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 496/990] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:06.437277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93678979-2d13-4149-a92f-96a035b67a5e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:06.437667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93678979-2d13-4149-a92f-96a035b67a5e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:06.437867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93678979-2d13-4149-a92f-96a035b67a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:06.445477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:06.457491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-93678979-2d13-4149-a92f-96a035b67a5e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:06.459190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-93678979-2d13-4149-a92f-96a035b67a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:06.530705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 495/991] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:06 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:06.552485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890b266-9cba-4198-83a1-1d9901d59420 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:06.615007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:06.672132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0890b266-9cba-4198-83a1-1d9901d59420 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:06.711249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf229f1-5275-4c06-9ba4-07897dd9c781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:06.711823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf229f1-5275-4c06-9ba4-07897dd9c781 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:06.712094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf229f1-5275-4c06-9ba4-07897dd9c781 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:06.712381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf229f1-5275-4c06-9ba4-07897dd9c781 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:06.713018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf229f1-5275-4c06-9ba4-07897dd9c781 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:06.714422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf229f1-5275-4c06-9ba4-07897dd9c781 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7cf229f1-5275-4c06-9ba4-07897dd9c781 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:06.724102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:06.725110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:06.731115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 497/992] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Jul 26 07:13:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:06.733672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93678979-2d13-4149-a92f-96a035b67a5e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:06.837439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93678979-2d13-4149-a92f-96a035b67a5e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:06.873260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42115a2b-a69a-4688-8927-190247077bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:06.874236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42115a2b-a69a-4688-8927-190247077bbe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:06.874735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42115a2b-a69a-4688-8927-190247077bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:06.875186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42115a2b-a69a-4688-8927-190247077bbe None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:06.876057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42115a2b-a69a-4688-8927-190247077bbe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:06.876554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42115a2b-a69a-4688-8927-190247077bbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:06.878147 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42115a2b-a69a-4688-8927-190247077bbe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:06.879560 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:06.885012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 496/993] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jul 26 07:13:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:06.891135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf229f1-5275-4c06-9ba4-07897dd9c781 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:06.993492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf229f1-5275-4c06-9ba4-07897dd9c781 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:07.012237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:07.030328 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4744ef-03c6-45d8-8439-57fae5a625bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:07.030970 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4744ef-03c6-45d8-8439-57fae5a625bc None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:07.031250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4744ef-03c6-45d8-8439-57fae5a625bc None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:07.031529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4744ef-03c6-45d8-8439-57fae5a625bc None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:07.032208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4744ef-03c6-45d8-8439-57fae5a625bc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:07.032513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4744ef-03c6-45d8-8439-57fae5a625bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:07.033909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4744ef-03c6-45d8-8439-57fae5a625bc None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:07.043920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 498/994] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Jul 26 07:13:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:07.045076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42115a2b-a69a-4688-8927-190247077bbe None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:07.148823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42115a2b-a69a-4688-8927-190247077bbe None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:07.183299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb55898-09ad-4099-bd8b-61f11fa8aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:07.183936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb55898-09ad-4099-bd8b-61f11fa8aa56 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:07.184219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb55898-09ad-4099-bd8b-61f11fa8aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:07.184489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb55898-09ad-4099-bd8b-61f11fa8aa56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:07.185201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb55898-09ad-4099-bd8b-61f11fa8aa56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:07.186605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb55898-09ad-4099-bd8b-61f11fa8aa56 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:07.213774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 497/995] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:13:07 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:13:07.364476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4c06-04c3-4db1-a093-c4e5137cd343 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:07.364925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4c06-04c3-4db1-a093-c4e5137cd343 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:07.365401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184d4c06-04c3-4db1-a093-c4e5137cd343 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:07.368103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 499/996] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:07.376648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e49432-98c9-4ce8-abf6-30cd6524af01 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:07.377168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e49432-98c9-4ce8-abf6-30cd6524af01 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:07.377621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e49432-98c9-4ce8-abf6-30cd6524af01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:07.407899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-37e49432-98c9-4ce8-abf6-30cd6524af01 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:07.409973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37e49432-98c9-4ce8-abf6-30cd6524af01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:07.489340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:07.511905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 498/997] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:07 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:07.534227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4c06-04c3-4db1-a093-c4e5137cd343 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:07.644016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184d4c06-04c3-4db1-a093-c4e5137cd343 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:07.680264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37eb2b-4461-407c-bdf7-9cd85d163f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:07.680915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37eb2b-4461-407c-bdf7-9cd85d163f3f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:07.681194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37eb2b-4461-407c-bdf7-9cd85d163f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:07.681468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37eb2b-4461-407c-bdf7-9cd85d163f3f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:07.683537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37eb2b-4461-407c-bdf7-9cd85d163f3f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:07.683537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37eb2b-4461-407c-bdf7-9cd85d163f3f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-bb37eb2b-4461-407c-bdf7-9cd85d163f3f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:07.693002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:07.694060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 07:13:07.694946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:07.697829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 500/998] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:13:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:07.700812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e49432-98c9-4ce8-abf6-30cd6524af01 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:07.809254 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e49432-98c9-4ce8-abf6-30cd6524af01 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:07.846169 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b9a7a-a135-4ca7-92cc-00cd44ded9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:07.846808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b9a7a-a135-4ca7-92cc-00cd44ded9fd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:07.847157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b9a7a-a135-4ca7-92cc-00cd44ded9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:07.847394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b9a7a-a135-4ca7-92cc-00cd44ded9fd None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:07.848083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b9a7a-a135-4ca7-92cc-00cd44ded9fd None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:07.848391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b9a7a-a135-4ca7-92cc-00cd44ded9fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:07.849818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b9a7a-a135-4ca7-92cc-00cd44ded9fd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:07.855608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 499/999] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:13:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:07.861979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37eb2b-4461-407c-bdf7-9cd85d163f3f None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:07.914188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:07.972977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37eb2b-4461-407c-bdf7-9cd85d163f3f None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:08.010353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeb1b4f-6cf2-4cec-b160-cbe4cfabd8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:08.010987 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb1b4f-6cf2-4cec-b160-cbe4cfabd8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:08.011267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb1b4f-6cf2-4cec-b160-cbe4cfabd8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:08.011538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeb1b4f-6cf2-4cec-b160-cbe4cfabd8f9 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:08.012227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb1b4f-6cf2-4cec-b160-cbe4cfabd8f9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:08.012526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb1b4f-6cf2-4cec-b160-cbe4cfabd8f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:08.014055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeb1b4f-6cf2-4cec-b160-cbe4cfabd8f9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:08.025120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 501/1000] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:13:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:08.026176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b9a7a-a135-4ca7-92cc-00cd44ded9fd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:08.085525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:08.171989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b9a7a-a135-4ca7-92cc-00cd44ded9fd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:08.222886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cecd0-6390-41fc-a432-7ef71aba6e5e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:08.223231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cecd0-6390-41fc-a432-7ef71aba6e5e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:08.223433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cecd0-6390-41fc-a432-7ef71aba6e5e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:08.224948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 502/1001] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:13:08.227737 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf3368-bce7-48a3-92b1-079700189a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:08.228861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf3368-bce7-48a3-92b1-079700189a70 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:08.229354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf3368-bce7-48a3-92b1-079700189a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:08.229862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf3368-bce7-48a3-92b1-079700189a70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:08.229980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe487b7-5210-4876-940e-1f4e83d3ff90 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:08.230143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe487b7-5210-4876-940e-1f4e83d3ff90 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:08.230419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe487b7-5210-4876-940e-1f4e83d3ff90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:08.231176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf3368-bce7-48a3-92b1-079700189a70 None admin] RBAC: Adding query filter params (interface=public, service_id=460687b66f49486da7fd84534ffbac1c, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:08.231526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf3368-bce7-48a3-92b1-079700189a70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:08.234498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf3368-bce7-48a3-92b1-079700189a70 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:08.244346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 500/1002] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jul 26 07:13:08 2022] GET /identity/v3/endpoints?service_id=460687b66f49486da7fd84534ffbac1c&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:08.250263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7fe487b7-5210-4876-940e-1f4e83d3ff90 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:08.251757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7fe487b7-5210-4876-940e-1f4e83d3ff90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:08.253168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf3368-bce7-48a3-92b1-079700189a70 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:08.330261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 503/1003] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:08 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:13:08.352262 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe487b7-5210-4876-940e-1f4e83d3ff90 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:08.383539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf3368-bce7-48a3-92b1-079700189a70 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:08.399801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:08.400474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:08.425649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90108bd5-98fc-43ab-bf1e-257322f529a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:08.426231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90108bd5-98fc-43ab-bf1e-257322f529a6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:08.426511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90108bd5-98fc-43ab-bf1e-257322f529a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:08.426783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90108bd5-98fc-43ab-bf1e-257322f529a6 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:08.427448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90108bd5-98fc-43ab-bf1e-257322f529a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:08.428840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90108bd5-98fc-43ab-bf1e-257322f529a6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:08.453526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 501/1004] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jul 26 07:13:08 2022] GET /identity/v3/services => generated 1727 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:13:08.497554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe487b7-5210-4876-940e-1f4e83d3ff90 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:08.541032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f841f-6ada-4a6c-9ecf-03e3b98e607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:08.541721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f841f-6ada-4a6c-9ecf-03e3b98e607b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:08.542025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f841f-6ada-4a6c-9ecf-03e3b98e607b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:08.542300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f841f-6ada-4a6c-9ecf-03e3b98e607b None admin] PATH_INFO: `/v3/services/image` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:08.542953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f841f-6ada-4a6c-9ecf-03e3b98e607b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:08.544373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f841f-6ada-4a6c-9ecf-03e3b98e607b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-804f841f-6ada-4a6c-9ecf-03e3b98e607b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:08.554457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:08.555590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 07:13:08.556446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:08.557922 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 504/1005] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Jul 26 07:13:08 2022] GET /identity/v3/services/image => generated 86 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:08.562011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90108bd5-98fc-43ab-bf1e-257322f529a6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:08.684692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90108bd5-98fc-43ab-bf1e-257322f529a6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:08.724995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc314a-6c04-43e3-9477-9868dcc24656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:08.726023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc314a-6c04-43e3-9477-9868dcc24656 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:08.726490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc314a-6c04-43e3-9477-9868dcc24656 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:08.726971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc314a-6c04-43e3-9477-9868dcc24656 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:08.727883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc314a-6c04-43e3-9477-9868dcc24656 None admin] RBAC: Adding query filter params (name=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:08.728396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc314a-6c04-43e3-9477-9868dcc24656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:08.730080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc314a-6c04-43e3-9477-9868dcc24656 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:08.736922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 502/1006] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:08.742929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f841f-6ada-4a6c-9ecf-03e3b98e607b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:08.858220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f841f-6ada-4a6c-9ecf-03e3b98e607b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:08.861316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:08.899777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1bfa0-a536-4e9d-825d-886f42982630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:08.900472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bfa0-a536-4e9d-825d-886f42982630 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:08.900762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bfa0-a536-4e9d-825d-886f42982630 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:08.901042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1bfa0-a536-4e9d-825d-886f42982630 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:08.901754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1bfa0-a536-4e9d-825d-886f42982630 None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:08.902071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1bfa0-a536-4e9d-825d-886f42982630 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:08.903450 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1bfa0-a536-4e9d-825d-886f42982630 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:08.917794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 505/1007] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:09.905022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dbe9e-9f2e-409b-9e75-b1ade8f9716c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:09.905588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dbe9e-9f2e-409b-9e75-b1ade8f9716c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:09.905588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3dbe9e-9f2e-409b-9e75-b1ade8f9716c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:09.906997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 503/1008] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:09.913359 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cae74-206c-4413-9758-adc6f0d0698f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:09.913975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cae74-206c-4413-9758-adc6f0d0698f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:09.913975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16cae74-206c-4413-9758-adc6f0d0698f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:09.933851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a16cae74-206c-4413-9758-adc6f0d0698f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:09.935118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a16cae74-206c-4413-9758-adc6f0d0698f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:10.011119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 506/1009] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:09 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:10.034416 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dbe9e-9f2e-409b-9e75-b1ade8f9716c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:10.096724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:10.164730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3dbe9e-9f2e-409b-9e75-b1ade8f9716c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:10.209040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e75a2d-e04c-4663-aa93-313de8001fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:10.209874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e75a2d-e04c-4663-aa93-313de8001fbe None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:10.210291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e75a2d-e04c-4663-aa93-313de8001fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:10.210573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e75a2d-e04c-4663-aa93-313de8001fbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:10.211794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e75a2d-e04c-4663-aa93-313de8001fbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:10.213292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e75a2d-e04c-4663-aa93-313de8001fbe None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:10.222022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 504/1010] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:10.224938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cae74-206c-4413-9758-adc6f0d0698f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:10.395733 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:10.403612 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16cae74-206c-4413-9758-adc6f0d0698f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:10.480257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847392d8-4f64-44b8-a72e-58271cbd60f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:10.481286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847392d8-4f64-44b8-a72e-58271cbd60f5 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:10.481761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847392d8-4f64-44b8-a72e-58271cbd60f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:10.482189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847392d8-4f64-44b8-a72e-58271cbd60f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:10.483452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847392d8-4f64-44b8-a72e-58271cbd60f5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:10.486546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847392d8-4f64-44b8-a72e-58271cbd60f5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:10.492012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dbe0c-d728-4f7e-88a4-7afb9369e011 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:10.492287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dbe0c-d728-4f7e-88a4-7afb9369e011 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:10.492612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29dbe0c-d728-4f7e-88a4-7afb9369e011 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:10.497790 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 505/1011] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:10.501808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4a669-bc91-4094-a8e8-9fab2751bbdc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:10.502349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4a669-bc91-4094-a8e8-9fab2751bbdc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:10.502799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4a669-bc91-4094-a8e8-9fab2751bbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:10.530663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-10c4a669-bc91-4094-a8e8-9fab2751bbdc None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:10.532650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-10c4a669-bc91-4094-a8e8-9fab2751bbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:10.538060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 507/1012] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:13:10 2022] POST /identity/v3/projects => generated 311 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:13:10.673006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 506/1013] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:10 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:13:10.696339 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847392d8-4f64-44b8-a72e-58271cbd60f5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:10.836996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847392d8-4f64-44b8-a72e-58271cbd60f5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:10.896547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5028f5d6-7c1b-4a0d-8943-3c29665047e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:10.897634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028f5d6-7c1b-4a0d-8943-3c29665047e3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:10.898161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028f5d6-7c1b-4a0d-8943-3c29665047e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:10.898619 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5028f5d6-7c1b-4a0d-8943-3c29665047e3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:10.899745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028f5d6-7c1b-4a0d-8943-3c29665047e3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:10.902216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5028f5d6-7c1b-4a0d-8943-3c29665047e3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5028f5d6-7c1b-4a0d-8943-3c29665047e3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:10.919702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:10.922714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:10.925086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 508/1014] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:13:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:10.932734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4a669-bc91-4094-a8e8-9fab2751bbdc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:10.973865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8dc7e-a6ba-40e0-94b8-142f3668fec3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:10.974136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8dc7e-a6ba-40e0-94b8-142f3668fec3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:10.974276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a8dc7e-a6ba-40e0-94b8-142f3668fec3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:10.977347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 509/1015] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:13:10.983890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98bec4-81f0-48b4-9821-11fa64011f47 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:10.984777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98bec4-81f0-48b4-9821-11fa64011f47 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:10.985693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea98bec4-81f0-48b4-9821-11fa64011f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:11.007428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ea98bec4-81f0-48b4-9821-11fa64011f47 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:11.009006 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea98bec4-81f0-48b4-9821-11fa64011f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:11.009211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:11.022717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:11.097589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 510/1016] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:10 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:11.110488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4a669-bc91-4094-a8e8-9fab2751bbdc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:11.120608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98bec4-81f0-48b4-9821-11fa64011f47 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:11.155262 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b61810-6cd1-4da0-9c2b-5e992e13f917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:11.156205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61810-6cd1-4da0-9c2b-5e992e13f917 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:11.156629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61810-6cd1-4da0-9c2b-5e992e13f917 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:11.157149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b61810-6cd1-4da0-9c2b-5e992e13f917 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:11.158052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b61810-6cd1-4da0-9c2b-5e992e13f917 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:11.158555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b61810-6cd1-4da0-9c2b-5e992e13f917 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:11.160382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b61810-6cd1-4da0-9c2b-5e992e13f917 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:11.167360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 507/1017] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:13:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:11.173167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b61810-6cd1-4da0-9c2b-5e992e13f917 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:11.263672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea98bec4-81f0-48b4-9821-11fa64011f47 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:11.322016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54de19a-2ac1-4778-9a34-9baa2e5b1955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:11.322677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de19a-2ac1-4778-9a34-9baa2e5b1955 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:11.322961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de19a-2ac1-4778-9a34-9baa2e5b1955 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:11.323237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de19a-2ac1-4778-9a34-9baa2e5b1955 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:11.323980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54de19a-2ac1-4778-9a34-9baa2e5b1955 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:11.325592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54de19a-2ac1-4778-9a34-9baa2e5b1955 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:11.337369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b61810-6cd1-4da0-9c2b-5e992e13f917 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:11.341160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:11.354442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 511/1018] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:13:11 2022] POST /identity/v3/services => generated 242 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:13:11.425942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb570055-e1d2-4929-bcb6-f52bb736cd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:11.429045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570055-e1d2-4929-bcb6-f52bb736cd84 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:11.429045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570055-e1d2-4929-bcb6-f52bb736cd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:11.429045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb570055-e1d2-4929-bcb6-f52bb736cd84 None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:11.429045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb570055-e1d2-4929-bcb6-f52bb736cd84 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:11.429646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb570055-e1d2-4929-bcb6-f52bb736cd84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:11.432028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb570055-e1d2-4929-bcb6-f52bb736cd84 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:11.453824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 508/1019] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Jul 26 07:13:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:11.456931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54de19a-2ac1-4778-9a34-9baa2e5b1955 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:11.465236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:11.655701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54de19a-2ac1-4778-9a34-9baa2e5b1955 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:11.718299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f8318-2fce-425b-9910-ba42a7fc4636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:11.719295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f8318-2fce-425b-9910-ba42a7fc4636 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:11.719783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f8318-2fce-425b-9910-ba42a7fc4636 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:11.720271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f8318-2fce-425b-9910-ba42a7fc4636 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:11.721663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f8318-2fce-425b-9910-ba42a7fc4636 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:11.724346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f8318-2fce-425b-9910-ba42a7fc4636 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:11.761866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 512/1020] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:13:11 2022] POST /identity/v3/endpoints => generated 334 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:13:13.431628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0aefd1-5b26-41d8-bf1e-d7821baa73c0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:13.432512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0aefd1-5b26-41d8-bf1e-d7821baa73c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:13.432512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0aefd1-5b26-41d8-bf1e-d7821baa73c0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:13.438299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 509/1021] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:13.440347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98993d8b-59ce-4eac-ab27-023aac141e87 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:13.440347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98993d8b-59ce-4eac-ab27-023aac141e87 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:13.441252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98993d8b-59ce-4eac-ab27-023aac141e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:13.477556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-98993d8b-59ce-4eac-ab27-023aac141e87 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:13.479242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98993d8b-59ce-4eac-ab27-023aac141e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:13.502283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315d317-1257-4cff-a2d9-0bcb800eb2b6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:13.502283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315d317-1257-4cff-a2d9-0bcb800eb2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:13.502283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9315d317-1257-4cff-a2d9-0bcb800eb2b6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:13.508083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 510/1022] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:13.509528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbc5b4-fa4b-447a-9fcd-3e9e8bcb386b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:13.512863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbc5b4-fa4b-447a-9fcd-3e9e8bcb386b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:13.512863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbc5b4-fa4b-447a-9fcd-3e9e8bcb386b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:13.535205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7cdbc5b4-fa4b-447a-9fcd-3e9e8bcb386b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:13.536451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7cdbc5b4-fa4b-447a-9fcd-3e9e8bcb386b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:13.567853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:13.602212 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 513/1023] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:13.618139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98993d8b-59ce-4eac-ab27-023aac141e87 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:13.626164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 511/1024] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:13.650616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbc5b4-fa4b-447a-9fcd-3e9e8bcb386b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:13.774914 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98993d8b-59ce-4eac-ab27-023aac141e87 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:13.810234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:13.825909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbc5b4-fa4b-447a-9fcd-3e9e8bcb386b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:13.828015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9555e5-e5ce-4f53-b9f9-057d51036e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:13.828015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9555e5-e5ce-4f53-b9f9-057d51036e88 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:13.828432 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9555e5-e5ce-4f53-b9f9-057d51036e88 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:13.828778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9555e5-e5ce-4f53-b9f9-057d51036e88 None admin] PATH_INFO: `/v3/services/image` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:13.829602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9555e5-e5ce-4f53-b9f9-057d51036e88 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:13.831257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9555e5-e5ce-4f53-b9f9-057d51036e88 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ff9555e5-e5ce-4f53-b9f9-057d51036e88 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:13.841137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:13.843116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:13.844925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 514/1025] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Jul 26 07:13:13 2022] GET /identity/v3/services/image => generated 86 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:13.851038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9555e5-e5ce-4f53-b9f9-057d51036e88 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:13.905731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683654da-d1ab-4753-92a5-54df90911b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:13.907206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683654da-d1ab-4753-92a5-54df90911b21 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:13.907696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683654da-d1ab-4753-92a5-54df90911b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:13.908183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683654da-d1ab-4753-92a5-54df90911b21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:13.910367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683654da-d1ab-4753-92a5-54df90911b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:13.912529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:13.912663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683654da-d1ab-4753-92a5-54df90911b21 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:13.917794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 512/1026] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:13.921870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683654da-d1ab-4753-92a5-54df90911b21 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:14.004578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9555e5-e5ce-4f53-b9f9-057d51036e88 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:14.065509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6af8c-9ddb-41a0-9457-d2ca11336270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:14.066506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6af8c-9ddb-41a0-9457-d2ca11336270 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:14.066942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6af8c-9ddb-41a0-9457-d2ca11336270 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:14.067378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a6af8c-9ddb-41a0-9457-d2ca11336270 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:14.068532 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6af8c-9ddb-41a0-9457-d2ca11336270 None admin] RBAC: Adding query filter params (name=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:14.069015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6af8c-9ddb-41a0-9457-d2ca11336270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:14.071605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a6af8c-9ddb-41a0-9457-d2ca11336270 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:14.080752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 515/1027] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:14.089194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6af8c-9ddb-41a0-9457-d2ca11336270 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:14.093181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683654da-d1ab-4753-92a5-54df90911b21 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:14.154226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003f5a2-e8a3-43fe-9c8a-7ce0b02dc84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:14.155210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003f5a2-e8a3-43fe-9c8a-7ce0b02dc84a None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:14.155672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003f5a2-e8a3-43fe-9c8a-7ce0b02dc84a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:14.156130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003f5a2-e8a3-43fe-9c8a-7ce0b02dc84a None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:14.157411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003f5a2-e8a3-43fe-9c8a-7ce0b02dc84a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:14.159525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1003f5a2-e8a3-43fe-9c8a-7ce0b02dc84a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:14.201806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 513/1028] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:13:13 2022] POST /identity/v3/users => generated 285 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:13:14.271204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a6af8c-9ddb-41a0-9457-d2ca11336270 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:14.299131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:14.340570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff88183-4bdc-4657-8426-ab7c233753eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:14.341611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff88183-4bdc-4657-8426-ab7c233753eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:14.342179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff88183-4bdc-4657-8426-ab7c233753eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:14.342684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff88183-4bdc-4657-8426-ab7c233753eb None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:14.344024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff88183-4bdc-4657-8426-ab7c233753eb None admin] RBAC: Adding query filter params (type=image) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:14.344565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff88183-4bdc-4657-8426-ab7c233753eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:14.347561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff88183-4bdc-4657-8426-ab7c233753eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:14.357613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 516/1029] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:14.367474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003f5a2-e8a3-43fe-9c8a-7ce0b02dc84a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:14.390321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:14.517903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003f5a2-e8a3-43fe-9c8a-7ce0b02dc84a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:14.558455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb5041a-946c-456d-825a-a13fbd284f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:14.559131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5041a-946c-456d-825a-a13fbd284f80 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:14.559506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5041a-946c-456d-825a-a13fbd284f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:14.559795 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb5041a-946c-456d-825a-a13fbd284f80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:14.560615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5041a-946c-456d-825a-a13fbd284f80 None admin] RBAC: Adding query filter params (interface=public, service_id=7cb3f74d69b44b01a45807ea7ef2318b, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:14.560928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5041a-946c-456d-825a-a13fbd284f80 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:14.562426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb5041a-946c-456d-825a-a13fbd284f80 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:14.568888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 514/1030] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jul 26 07:13:14 2022] GET /identity/v3/endpoints?service_id=7cb3f74d69b44b01a45807ea7ef2318b&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:14.574694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff88183-4bdc-4657-8426-ab7c233753eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:14.698742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff88183-4bdc-4657-8426-ab7c233753eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:14.737936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4743d-d1a7-44ab-b946-cf377419db86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:14.738634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4743d-d1a7-44ab-b946-cf377419db86 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:14.738914 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4743d-d1a7-44ab-b946-cf377419db86 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:14.739196 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4743d-d1a7-44ab-b946-cf377419db86 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:14.739855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4743d-d1a7-44ab-b946-cf377419db86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:14.741250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4743d-d1a7-44ab-b946-cf377419db86 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:14.749162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 517/1031] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jul 26 07:13:14 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:13:15.016276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797e87b-3dba-494c-a426-7a80d6fea824 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:15.016660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797e87b-3dba-494c-a426-7a80d6fea824 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:15.016916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797e87b-3dba-494c-a426-7a80d6fea824 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:15.018462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 515/1032] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:15.025258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c7676-c1fb-4731-9231-561a0a21a0b6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:15.025504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c7676-c1fb-4731-9231-561a0a21a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:15.026024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c7676-c1fb-4731-9231-561a0a21a0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:15.046636 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bc5c7676-c1fb-4731-9231-561a0a21a0b6 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:15.047912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bc5c7676-c1fb-4731-9231-561a0a21a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:15.091690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:15.133792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 518/1033] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:15.154037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797e87b-3dba-494c-a426-7a80d6fea824 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:15.248557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:15.273145 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797e87b-3dba-494c-a426-7a80d6fea824 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:15.311715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da73aaf-951b-4dc0-ae7b-1ea9c39ba939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:15.312370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da73aaf-951b-4dc0-ae7b-1ea9c39ba939 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:15.312650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da73aaf-951b-4dc0-ae7b-1ea9c39ba939 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:15.312919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da73aaf-951b-4dc0-ae7b-1ea9c39ba939 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:15.324975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da73aaf-951b-4dc0-ae7b-1ea9c39ba939 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6da73aaf-951b-4dc0-ae7b-1ea9c39ba939 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:13:15.330067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:15.337480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:15.338409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 516/1034] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jul 26 07:13:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:15.340920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c7676-c1fb-4731-9231-561a0a21a0b6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:15.453995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c7676-c1fb-4731-9231-561a0a21a0b6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:15.493904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e983c7-a6d3-4e3e-8a69-4cc4e5226df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:15.494557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e983c7-a6d3-4e3e-8a69-4cc4e5226df5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:15.494840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e983c7-a6d3-4e3e-8a69-4cc4e5226df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:15.495120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e983c7-a6d3-4e3e-8a69-4cc4e5226df5 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:15.495870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e983c7-a6d3-4e3e-8a69-4cc4e5226df5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:15.496189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e983c7-a6d3-4e3e-8a69-4cc4e5226df5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:15.497579 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e983c7-a6d3-4e3e-8a69-4cc4e5226df5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:15.507181 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 519/1035] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jul 26 07:13:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:15.512931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da73aaf-951b-4dc0-ae7b-1ea9c39ba939 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:15.622695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da73aaf-951b-4dc0-ae7b-1ea9c39ba939 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:15.663945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c954dba-38cf-4d3f-b9e3-f621047f4389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:15.664613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c954dba-38cf-4d3f-b9e3-f621047f4389 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:15.664896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c954dba-38cf-4d3f-b9e3-f621047f4389 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:15.665174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c954dba-38cf-4d3f-b9e3-f621047f4389 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:15.674993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:15.678976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c954dba-38cf-4d3f-b9e3-f621047f4389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:15.680921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c954dba-38cf-4d3f-b9e3-f621047f4389 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-7c954dba-38cf-4d3f-b9e3-f621047f4389 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:15.698335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:15.699312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:15.700204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:13:15.700204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:15.700204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:15.700204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:15.702078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 517/1036] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:15.705509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e983c7-a6d3-4e3e-8a69-4cc4e5226df5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:15.745544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:15.856074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e983c7-a6d3-4e3e-8a69-4cc4e5226df5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:15.919893 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baff84-4a93-493d-ba6a-c2bb377d87d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:15.920776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baff84-4a93-493d-ba6a-c2bb377d87d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:15.921318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baff84-4a93-493d-ba6a-c2bb377d87d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:15.921894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55baff84-4a93-493d-ba6a-c2bb377d87d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:15.923231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baff84-4a93-493d-ba6a-c2bb377d87d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:15.924811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baff84-4a93-493d-ba6a-c2bb377d87d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:15.927300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55baff84-4a93-493d-ba6a-c2bb377d87d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:15.949490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 520/1037] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:15.959652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c954dba-38cf-4d3f-b9e3-f621047f4389 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:16.139288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c954dba-38cf-4d3f-b9e3-f621047f4389 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:16.181846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7629c75-3b93-411f-890c-a9cd311e24d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:16.182866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7629c75-3b93-411f-890c-a9cd311e24d7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:16.183356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7629c75-3b93-411f-890c-a9cd311e24d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:16.183834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7629c75-3b93-411f-890c-a9cd311e24d7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:16.194148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7629c75-3b93-411f-890c-a9cd311e24d7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:16.196022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7629c75-3b93-411f-890c-a9cd311e24d7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c7629c75-3b93-411f-890c-a9cd311e24d7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:13:16.208855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:16.209949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:16.210907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:16.210907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:16.210907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:13:16.210907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:13:16.210907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:13:16.210907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:13:16.210907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:13:16.210907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:13:16.210907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:13:16.210907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:16.212274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 518/1038] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jul 26 07:13:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:16.218777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baff84-4a93-493d-ba6a-c2bb377d87d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:16.327133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:16.335515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55baff84-4a93-493d-ba6a-c2bb377d87d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:16.373584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f9a02-d737-42f6-80c3-b7606bcd4145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:16.374243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f9a02-d737-42f6-80c3-b7606bcd4145 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:16.374519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f9a02-d737-42f6-80c3-b7606bcd4145 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:16.374789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4f9a02-d737-42f6-80c3-b7606bcd4145 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:16.375597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f9a02-d737-42f6-80c3-b7606bcd4145 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:16.375897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f9a02-d737-42f6-80c3-b7606bcd4145 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:16.377345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4f9a02-d737-42f6-80c3-b7606bcd4145 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:16.405575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 521/1039] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jul 26 07:13:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:16.411552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7629c75-3b93-411f-890c-a9cd311e24d7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:16.480573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:16.529264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7629c75-3b93-411f-890c-a9cd311e24d7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:16.564055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb59d61-8d6f-441d-818f-9de17a82db06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:16.564688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb59d61-8d6f-441d-818f-9de17a82db06 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:16.564962 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb59d61-8d6f-441d-818f-9de17a82db06 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:16.565230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb59d61-8d6f-441d-818f-9de17a82db06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:16.633359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb59d61-8d6f-441d-818f-9de17a82db06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:16.634859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb59d61-8d6f-441d-818f-9de17a82db06 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:16.692112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:16.742411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 519/1040] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:16 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:13:16.748203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f9a02-d737-42f6-80c3-b7606bcd4145 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:16.812303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f35b60-0cd7-4672-b3c1-251f9d9207fd None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:16.812731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f35b60-0cd7-4672-b3c1-251f9d9207fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:16.812844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f35b60-0cd7-4672-b3c1-251f9d9207fd None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:16.814209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 520/1041] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:13:16.820093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71c9fb-58e9-4409-97f0-890649068a5f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:16.820353 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71c9fb-58e9-4409-97f0-890649068a5f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:16.820642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e71c9fb-58e9-4409-97f0-890649068a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:16.839017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5e71c9fb-58e9-4409-97f0-890649068a5f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:16.840203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5e71c9fb-58e9-4409-97f0-890649068a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:16.865147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4f9a02-d737-42f6-80c3-b7606bcd4145 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:16.900542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748b4ce-fe3e-476c-9d7f-fca5ab1734eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:16.901194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748b4ce-fe3e-476c-9d7f-fca5ab1734eb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:16.901482 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748b4ce-fe3e-476c-9d7f-fca5ab1734eb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:16.901770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e748b4ce-fe3e-476c-9d7f-fca5ab1734eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:16.906897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 521/1042] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:13:16.912249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748b4ce-fe3e-476c-9d7f-fca5ab1734eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:16.913700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e748b4ce-fe3e-476c-9d7f-fca5ab1734eb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e748b4ce-fe3e-476c-9d7f-fca5ab1734eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:16.928546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:16.929497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:16.930414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:13:16.930414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:16.930414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:16.930414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:16.930646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71c9fb-58e9-4409-97f0-890649068a5f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:16.934058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 522/1043] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:16.935332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748b4ce-fe3e-476c-9d7f-fca5ab1734eb None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:16.939020 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:17.016392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:17.036567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e71c9fb-58e9-4409-97f0-890649068a5f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:17.040838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e748b4ce-fe3e-476c-9d7f-fca5ab1734eb None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:17.072025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afc598-2cc4-473a-a1e4-e4e18d9af6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:17.072669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afc598-2cc4-473a-a1e4-e4e18d9af6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.072953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afc598-2cc4-473a-a1e4-e4e18d9af6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.073226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9afc598-2cc4-473a-a1e4-e4e18d9af6e4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.073917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afc598-2cc4-473a-a1e4-e4e18d9af6e4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:17.075303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9afc598-2cc4-473a-a1e4-e4e18d9af6e4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:17.075940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988140f6-2835-4d26-853b-7c9d2398b55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:17.076571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988140f6-2835-4d26-853b-7c9d2398b55a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.076855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988140f6-2835-4d26-853b-7c9d2398b55a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.077132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988140f6-2835-4d26-853b-7c9d2398b55a None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.077853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988140f6-2835-4d26-853b-7c9d2398b55a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:17.078170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988140f6-2835-4d26-853b-7c9d2398b55a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:17.079511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988140f6-2835-4d26-853b-7c9d2398b55a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d9afc598-2cc4-473a-a1e4-e4e18d9af6e4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:17.084905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:17.086186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 26 07:13:17.087036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:17.091400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 522/1044] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Jul 26 07:13:16 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:17.092119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afc598-2cc4-473a-a1e4-e4e18d9af6e4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:17.094039 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 523/1045] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:17.100293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988140f6-2835-4d26-853b-7c9d2398b55a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:17.174754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:17.198461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9afc598-2cc4-473a-a1e4-e4e18d9af6e4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:17.214175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988140f6-2835-4d26-853b-7c9d2398b55a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:17.231267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3117b62b-3678-48f3-82b4-46d0d91c2c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:17.231900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117b62b-3678-48f3-82b4-46d0d91c2c5d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.232202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117b62b-3678-48f3-82b4-46d0d91c2c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.232472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3117b62b-3678-48f3-82b4-46d0d91c2c5d None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.233156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117b62b-3678-48f3-82b4-46d0d91c2c5d None admin] RBAC: Adding query filter params (name=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:17.233458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117b62b-3678-48f3-82b4-46d0d91c2c5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:17.235228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3117b62b-3678-48f3-82b4-46d0d91c2c5d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:17.240374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 523/1046] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:17.241555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a63d2-2d49-4d21-bed5-ca5068956c5e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.241875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a63d2-2d49-4d21-bed5-ca5068956c5e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.242151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369a63d2-2d49-4d21-bed5-ca5068956c5e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.243379 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 524/1047] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:17.246555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a63d2-2d49-4d21-bed5-ca5068956c5e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:17.250089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58cb64-e043-456a-b14d-0de16d9b8acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:17.250744 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58cb64-e043-456a-b14d-0de16d9b8acd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.251022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58cb64-e043-456a-b14d-0de16d9b8acd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.251291 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58cb64-e043-456a-b14d-0de16d9b8acd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.258367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58cb64-e043-456a-b14d-0de16d9b8acd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:17.259551 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:17.259843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58cb64-e043-456a-b14d-0de16d9b8acd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ce58cb64-e043-456a-b14d-0de16d9b8acd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:13:17.273242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:17.274337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:17.275649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:13:17.275649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:13:17.275649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:13:17.275649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:13:17.275649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:13:17.275649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:13:17.275649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:13:17.275649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:17.275649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 524/1048] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:17.275649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8592b-168e-4a30-993b-53842dd47d4e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.276193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8592b-168e-4a30-993b-53842dd47d4e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.276193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8592b-168e-4a30-993b-53842dd47d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.294064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-26d8592b-168e-4a30-993b-53842dd47d4e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:17.295266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-26d8592b-168e-4a30-993b-53842dd47d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:17.352998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369a63d2-2d49-4d21-bed5-ca5068956c5e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:17.362693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 525/1049] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:17.363670 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8592b-168e-4a30-993b-53842dd47d4e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:17.391661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29d9b4-be1c-485b-b5de-90c8d03e531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:17.392298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29d9b4-be1c-485b-b5de-90c8d03e531c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.392580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29d9b4-be1c-485b-b5de-90c8d03e531c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.392852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be29d9b4-be1c-485b-b5de-90c8d03e531c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.393523 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29d9b4-be1c-485b-b5de-90c8d03e531c None admin] RBAC: Adding query filter params (type=image) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:17.393864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29d9b4-be1c-485b-b5de-90c8d03e531c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:17.395351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be29d9b4-be1c-485b-b5de-90c8d03e531c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:17.404808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 525/1050] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:17.404808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29d9b4-be1c-485b-b5de-90c8d03e531c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:17.433695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:17.479066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:17.481078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8592b-168e-4a30-993b-53842dd47d4e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:17.505846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be29d9b4-be1c-485b-b5de-90c8d03e531c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:17.516673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230abce-29da-44a8-a43d-914a49772853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:17.517309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230abce-29da-44a8-a43d-914a49772853 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.517560 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230abce-29da-44a8-a43d-914a49772853 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.517878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7230abce-29da-44a8-a43d-914a49772853 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.518638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230abce-29da-44a8-a43d-914a49772853 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:17.518939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230abce-29da-44a8-a43d-914a49772853 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:17.520450 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7230abce-29da-44a8-a43d-914a49772853 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:17.532040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 526/1051] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:13:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:17.532746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230abce-29da-44a8-a43d-914a49772853 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:17.541210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ca739-9694-4192-a2b5-0a0546c065ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:17.542305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ca739-9694-4192-a2b5-0a0546c065ce None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.542625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ca739-9694-4192-a2b5-0a0546c065ce None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.542857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ca739-9694-4192-a2b5-0a0546c065ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.554921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ca739-9694-4192-a2b5-0a0546c065ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-109ca739-9694-4192-a2b5-0a0546c065ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:13:17.560290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:17.561252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:17.562181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 526/1052] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:13:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:17.562890 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ca739-9694-4192-a2b5-0a0546c065ce None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:17.639531 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7230abce-29da-44a8-a43d-914a49772853 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:17.653618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:17.670024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ca739-9694-4192-a2b5-0a0546c065ce None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:17.674554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ba84b-75b1-42b2-b642-a00c688f922a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:17.675175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ba84b-75b1-42b2-b642-a00c688f922a None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.675454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ba84b-75b1-42b2-b642-a00c688f922a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.675766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ba84b-75b1-42b2-b642-a00c688f922a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.676476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ba84b-75b1-42b2-b642-a00c688f922a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:17.677871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ba84b-75b1-42b2-b642-a00c688f922a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:17.698446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 527/1053] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:13:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:13:17.699275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ba84b-75b1-42b2-b642-a00c688f922a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:17.707395 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:17.716928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4f901-eeca-4b66-bfdc-526c753d4dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:17.717730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4f901-eeca-4b66-bfdc-526c753d4dbc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.718048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4f901-eeca-4b66-bfdc-526c753d4dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.718378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4f901-eeca-4b66-bfdc-526c753d4dbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.719214 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4f901-eeca-4b66-bfdc-526c753d4dbc None admin] RBAC: Adding query filter params (role.id=06dfeb3edc3b4716b86a7f7be42d71b1, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=6d50427014064f099af87932c77a849a) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:17.719523 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4f901-eeca-4b66-bfdc-526c753d4dbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:17.721274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4f901-eeca-4b66-bfdc-526c753d4dbc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:17.728510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 527/1054] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:13:17 2022] GET /identity/v3/role_assignments?user.id=6d50427014064f099af87932c77a849a&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=06dfeb3edc3b4716b86a7f7be42d71b1 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:17.829924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ba84b-75b1-42b2-b642-a00c688f922a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:17.875892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ba520-6090-46ae-a7fb-801e1259a752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:17.877114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ba520-6090-46ae-a7fb-801e1259a752 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:17.877114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ba520-6090-46ae-a7fb-801e1259a752 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:17.877114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ba520-6090-46ae-a7fb-801e1259a752 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:17.878561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ba520-6090-46ae-a7fb-801e1259a752 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:17.878561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ba520-6090-46ae-a7fb-801e1259a752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:17.879865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ba520-6090-46ae-a7fb-801e1259a752 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:17.890692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 528/1055] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:13:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:17.897010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4f901-eeca-4b66-bfdc-526c753d4dbc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:18.004205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4f901-eeca-4b66-bfdc-526c753d4dbc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:18.038854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c71e3-0780-4cf3-93bf-6c72ea3cb741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:18.039519 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c71e3-0780-4cf3-93bf-6c72ea3cb741 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:18.039802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c71e3-0780-4cf3-93bf-6c72ea3cb741 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:18.040072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c71e3-0780-4cf3-93bf-6c72ea3cb741 None admin] PATH_INFO: `/v3/users/9f52e136264d473fac028910e57f94f1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:18.055195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c71e3-0780-4cf3-93bf-6c72ea3cb741 None admin] RBAC: Authorizing `identity:get_user(user_id=9f52e136264d473fac028910e57f94f1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:18.056659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545c71e3-0780-4cf3-93bf-6c72ea3cb741 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:18.058620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 528/1056] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:17 2022] GET /identity/v3/users/9f52e136264d473fac028910e57f94f1 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:18.064313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ba520-6090-46ae-a7fb-801e1259a752 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:18.098647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:18.179420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ba520-6090-46ae-a7fb-801e1259a752 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:18.222792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502954b8-ebfa-42bf-b963-cdba118a4a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:18.223444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502954b8-ebfa-42bf-b963-cdba118a4a8e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:18.223724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502954b8-ebfa-42bf-b963-cdba118a4a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:18.223999 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502954b8-ebfa-42bf-b963-cdba118a4a8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:18.297365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502954b8-ebfa-42bf-b963-cdba118a4a8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:18.299207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502954b8-ebfa-42bf-b963-cdba118a4a8e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:18.330137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:18.402785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 529/1057] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 07:13:18.408469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c71e3-0780-4cf3-93bf-6c72ea3cb741 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:18.474161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:18.524774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545c71e3-0780-4cf3-93bf-6c72ea3cb741 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:18.558672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30d9dd-694e-487f-aa1a-000c0176d96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:18.559321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30d9dd-694e-487f-aa1a-000c0176d96d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:18.559604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30d9dd-694e-487f-aa1a-000c0176d96d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:18.559874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30d9dd-694e-487f-aa1a-000c0176d96d None admin] PATH_INFO: `/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:18.561040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30d9dd-694e-487f-aa1a-000c0176d96d None admin] RBAC: Authorizing `identity:get_project(project_id=a54bad337df54d0a81bdc2fc94a6c3e5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:18.562516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30d9dd-694e-487f-aa1a-000c0176d96d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:18.564356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 529/1058] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:13:18 2022] GET /identity/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:18.570587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502954b8-ebfa-42bf-b963-cdba118a4a8e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:18.670758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502954b8-ebfa-42bf-b963-cdba118a4a8e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:18.700132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:18.706522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179deb75-89b0-4150-a30b-64109802a742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:18.707179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179deb75-89b0-4150-a30b-64109802a742 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:18.707461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179deb75-89b0-4150-a30b-64109802a742 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:18.707726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179deb75-89b0-4150-a30b-64109802a742 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:18.710516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179deb75-89b0-4150-a30b-64109802a742 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=a54bad337df54d0a81bdc2fc94a6c3e5, user.id=9f52e136264d473fac028910e57f94f1) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:18.710516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179deb75-89b0-4150-a30b-64109802a742 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:18.710516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179deb75-89b0-4150-a30b-64109802a742 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:18.716357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 530/1059] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:13:18 2022] GET /identity/v3/role_assignments?user.id=9f52e136264d473fac028910e57f94f1&scope.project.id=a54bad337df54d0a81bdc2fc94a6c3e5&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:19.722302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd721d-e6ab-4824-a367-b2e7c9059487 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:19.722847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd721d-e6ab-4824-a367-b2e7c9059487 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:19.722847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bd721d-e6ab-4824-a367-b2e7c9059487 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:19.724359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 530/1060] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:19.729780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c73780-1e58-4bff-8bdd-0bba3880f8e2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:19.730245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c73780-1e58-4bff-8bdd-0bba3880f8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:19.730336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c73780-1e58-4bff-8bdd-0bba3880f8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:19.749127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d0c73780-1e58-4bff-8bdd-0bba3880f8e2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:19.750339 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0c73780-1e58-4bff-8bdd-0bba3880f8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:19.759825 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df1f30-21b3-4bb8-9f8c-7770c30eecac None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:19.760103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df1f30-21b3-4bb8-9f8c-7770c30eecac None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:19.760373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df1f30-21b3-4bb8-9f8c-7770c30eecac None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:19.761723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 531/1061] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:19.766863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99ffee-6e79-4903-9aaf-26b2e595c052 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:19.767148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99ffee-6e79-4903-9aaf-26b2e595c052 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:19.767436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee99ffee-6e79-4903-9aaf-26b2e595c052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:19.785479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ee99ffee-6e79-4903-9aaf-26b2e595c052 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:19.786758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ee99ffee-6e79-4903-9aaf-26b2e595c052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:19.837780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 531/1062] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:19.860802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c73780-1e58-4bff-8bdd-0bba3880f8e2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:19.874898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 532/1063] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:13:19.889237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99ffee-6e79-4903-9aaf-26b2e595c052 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:19.941130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:19.966794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c73780-1e58-4bff-8bdd-0bba3880f8e2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:19.993683 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:19.998736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee99ffee-6e79-4903-9aaf-26b2e595c052 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:20.004022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b8d77-766e-49a9-9105-0b010ae5dd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:20.004685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b8d77-766e-49a9-9105-0b010ae5dd4a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.004966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b8d77-766e-49a9-9105-0b010ae5dd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.005241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b8d77-766e-49a9-9105-0b010ae5dd4a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.016021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b8d77-766e-49a9-9105-0b010ae5dd4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e69b8d77-766e-49a9-9105-0b010ae5dd4a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:13:20.020896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:20.021867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:20.022723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 532/1064] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jul 26 07:13:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:20.028009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b8d77-766e-49a9-9105-0b010ae5dd4a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:20.036820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a45ff-048b-4180-ad1d-e06273aec5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:20.037459 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a45ff-048b-4180-ad1d-e06273aec5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.037753 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a45ff-048b-4180-ad1d-e06273aec5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.038027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a45ff-048b-4180-ad1d-e06273aec5d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.111018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a45ff-048b-4180-ad1d-e06273aec5d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:20.112511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a45ff-048b-4180-ad1d-e06273aec5d6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:20.153558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b8d77-766e-49a9-9105-0b010ae5dd4a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:20.179327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:20.201711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaef6c-0eff-441d-b36b-1ed3490281d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:20.202401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaef6c-0eff-441d-b36b-1ed3490281d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.202693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaef6c-0eff-441d-b36b-1ed3490281d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.202945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adaef6c-0eff-441d-b36b-1ed3490281d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.203687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaef6c-0eff-441d-b36b-1ed3490281d2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:20.203993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaef6c-0eff-441d-b36b-1ed3490281d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:20.205404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adaef6c-0eff-441d-b36b-1ed3490281d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:20.214936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 533/1065] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jul 26 07:13:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:20.220565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaef6c-0eff-441d-b36b-1ed3490281d2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:20.245230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 533/1066] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 07:13:20.250990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a45ff-048b-4180-ad1d-e06273aec5d6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:20.277007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:20.332296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adaef6c-0eff-441d-b36b-1ed3490281d2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:20.358279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a45ff-048b-4180-ad1d-e06273aec5d6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:20.370055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f6090-542e-4407-8491-360b11c9dfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:20.370701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f6090-542e-4407-8491-360b11c9dfad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.370984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f6090-542e-4407-8491-360b11c9dfad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.371257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f6090-542e-4407-8491-360b11c9dfad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.383384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f6090-542e-4407-8491-360b11c9dfad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:20.384858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f6090-542e-4407-8491-360b11c9dfad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:20.396283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074965e-f724-451d-9be8-28234f45173a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:20.396946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074965e-f724-451d-9be8-28234f45173a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.397221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074965e-f724-451d-9be8-28234f45173a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.397494 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1074965e-f724-451d-9be8-28234f45173a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-989f6090-542e-4407-8491-360b11c9dfad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:20.399935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:20.401008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:20.402069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:13:20.402069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:20.402069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:20.402069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:20.402069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 534/1067] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:20.406884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f6090-542e-4407-8491-360b11c9dfad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:20.412411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074965e-f724-451d-9be8-28234f45173a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:20.413923 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1074965e-f724-451d-9be8-28234f45173a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1074965e-f724-451d-9be8-28234f45173a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:20.433966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:20.435137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:20.436242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:13:20.436242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:20.436242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:20.436242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:20.436242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 534/1068] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:20.441326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074965e-f724-451d-9be8-28234f45173a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:20.445460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:20.523313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:20.526969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f6090-542e-4407-8491-360b11c9dfad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:20.557704 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1074965e-f724-451d-9be8-28234f45173a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:20.566820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10296a63-28ea-4aff-afff-f5ace002399d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:20.567792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10296a63-28ea-4aff-afff-f5ace002399d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.568076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10296a63-28ea-4aff-afff-f5ace002399d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.568364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10296a63-28ea-4aff-afff-f5ace002399d None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.569370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10296a63-28ea-4aff-afff-f5ace002399d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:20.569685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10296a63-28ea-4aff-afff-f5ace002399d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:20.571090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10296a63-28ea-4aff-afff-f5ace002399d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:20.584796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 535/1069] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:20.592062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10296a63-28ea-4aff-afff-f5ace002399d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:20.594050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a5731-53d4-4683-b854-9f8b4d4cebe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:20.594695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a5731-53d4-4683-b854-9f8b4d4cebe1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.594973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a5731-53d4-4683-b854-9f8b4d4cebe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.595242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a5731-53d4-4683-b854-9f8b4d4cebe1 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.595984 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a5731-53d4-4683-b854-9f8b4d4cebe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:20.596281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a5731-53d4-4683-b854-9f8b4d4cebe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:20.597826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a5731-53d4-4683-b854-9f8b4d4cebe1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:20.611491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 535/1070] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:20.704620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b283794-7b39-42fa-ac6f-e43e86576edb None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.704995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b283794-7b39-42fa-ac6f-e43e86576edb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.705179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b283794-7b39-42fa-ac6f-e43e86576edb None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.706842 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 536/1071] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:13:20.713115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6de7f8-14fe-4b2f-9e45-c2ce403099c3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.713499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6de7f8-14fe-4b2f-9e45-c2ce403099c3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.713662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6de7f8-14fe-4b2f-9e45-c2ce403099c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.722173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10296a63-28ea-4aff-afff-f5ace002399d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:20.733531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ad6de7f8-14fe-4b2f-9e45-c2ce403099c3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:20.735221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad6de7f8-14fe-4b2f-9e45-c2ce403099c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:20.769972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23d4b5-fa48-49b5-969a-8c89a5161149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:20.771293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d4b5-fa48-49b5-969a-8c89a5161149 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.771730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d4b5-fa48-49b5-969a-8c89a5161149 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.772011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23d4b5-fa48-49b5-969a-8c89a5161149 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.780691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23d4b5-fa48-49b5-969a-8c89a5161149 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:20.782605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23d4b5-fa48-49b5-969a-8c89a5161149 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-ae23d4b5-fa48-49b5-969a-8c89a5161149 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:13:20.797565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:20.798686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:20.800011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 536/1072] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jul 26 07:13:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:20.804693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23d4b5-fa48-49b5-969a-8c89a5161149 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:20.805077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:20.822446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 537/1073] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:13:20.841716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:20.844907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6de7f8-14fe-4b2f-9e45-c2ce403099c3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:20.952749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23d4b5-fa48-49b5-969a-8c89a5161149 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:20.977609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6de7f8-14fe-4b2f-9e45-c2ce403099c3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:20.995891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe77683-8c58-474d-a0e0-b0ebdddd60b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:20.996544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe77683-8c58-474d-a0e0-b0ebdddd60b0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:20.996826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe77683-8c58-474d-a0e0-b0ebdddd60b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:20.997108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe77683-8c58-474d-a0e0-b0ebdddd60b0 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:20.997948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe77683-8c58-474d-a0e0-b0ebdddd60b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:20.998257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe77683-8c58-474d-a0e0-b0ebdddd60b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:20.999702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe77683-8c58-474d-a0e0-b0ebdddd60b0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:21.016953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a37028-8039-493e-91c1-e8cef6141661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:21.017897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a37028-8039-493e-91c1-e8cef6141661 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:21.018013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 537/1074] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jul 26 07:13:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:21.018104 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a37028-8039-493e-91c1-e8cef6141661 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:21.018188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a37028-8039-493e-91c1-e8cef6141661 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:21.024685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe77683-8c58-474d-a0e0-b0ebdddd60b0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:21.031240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a37028-8039-493e-91c1-e8cef6141661 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-57a37028-8039-493e-91c1-e8cef6141661 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:13:21.036229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:21.037323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:21.038205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 538/1075] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:13:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:21.043431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a37028-8039-493e-91c1-e8cef6141661 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:21.102417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:21.117295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:21.140890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe77683-8c58-474d-a0e0-b0ebdddd60b0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:21.158321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a37028-8039-493e-91c1-e8cef6141661 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:21.181410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2105b3-20af-475f-acdd-212da3fe72ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:21.182104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2105b3-20af-475f-acdd-212da3fe72ad None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:21.182390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2105b3-20af-475f-acdd-212da3fe72ad None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:21.182665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2105b3-20af-475f-acdd-212da3fe72ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:21.193406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2105b3-20af-475f-acdd-212da3fe72ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:21.194912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2105b3-20af-475f-acdd-212da3fe72ad None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:21.202152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1d7f9-b003-41ad-a551-f1539862c8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:21.202789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1d7f9-b003-41ad-a551-f1539862c8e0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:21.203072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1d7f9-b003-41ad-a551-f1539862c8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:21.203343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1d7f9-b003-41ad-a551-f1539862c8e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:21.204065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1d7f9-b003-41ad-a551-f1539862c8e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:21.204368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1d7f9-b003-41ad-a551-f1539862c8e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:21.205769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1d7f9-b003-41ad-a551-f1539862c8e0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-eb2105b3-20af-475f-acdd-212da3fe72ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:21.211462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:21.212381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:21.213458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:13:21.213458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:21.213458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:21.213458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:21.213458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 538/1076] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:21.217217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 539/1077] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:13:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:21.218555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2105b3-20af-475f-acdd-212da3fe72ad None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:21.220972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1d7f9-b003-41ad-a551-f1539862c8e0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:21.328454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2105b3-20af-475f-acdd-212da3fe72ad None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:21.343069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1d7f9-b003-41ad-a551-f1539862c8e0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:21.357717 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:21.361406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:21.368454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3930e7-39a9-4897-84c3-5ba05429a3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:21.369083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3930e7-39a9-4897-84c3-5ba05429a3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:21.369358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3930e7-39a9-4897-84c3-5ba05429a3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:21.369629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3930e7-39a9-4897-84c3-5ba05429a3e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:21.370377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3930e7-39a9-4897-84c3-5ba05429a3e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:21.370678 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3930e7-39a9-4897-84c3-5ba05429a3e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:21.372070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3930e7-39a9-4897-84c3-5ba05429a3e6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:21.384379 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 539/1078] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:21.384571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f5b09-a385-4c9c-85ef-3cb1e49bbbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:21.386794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f5b09-a385-4c9c-85ef-3cb1e49bbbee None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:21.386794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f5b09-a385-4c9c-85ef-3cb1e49bbbee None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:21.386794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f5b09-a385-4c9c-85ef-3cb1e49bbbee None admin] PATH_INFO: `/v3/users/9f52e136264d473fac028910e57f94f1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:21.387160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f5b09-a385-4c9c-85ef-3cb1e49bbbee None admin] RBAC: Authorizing `identity:get_user(user_id=9f52e136264d473fac028910e57f94f1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:21.388610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f5b09-a385-4c9c-85ef-3cb1e49bbbee None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:21.390266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3930e7-39a9-4897-84c3-5ba05429a3e6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:21.390385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 540/1079] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:21 2022] GET /identity/v3/users/9f52e136264d473fac028910e57f94f1 => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:21.396227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f5b09-a385-4c9c-85ef-3cb1e49bbbee None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:21.500695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3930e7-39a9-4897-84c3-5ba05429a3e6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:21.505441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f5b09-a385-4c9c-85ef-3cb1e49bbbee None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:21.540940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24884b2e-78b4-4c64-b929-11372d8ee93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:21.541595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24884b2e-78b4-4c64-b929-11372d8ee93e None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:21.541909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24884b2e-78b4-4c64-b929-11372d8ee93e None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:21.542185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24884b2e-78b4-4c64-b929-11372d8ee93e None admin] PATH_INFO: `/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:21.543332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24884b2e-78b4-4c64-b929-11372d8ee93e None admin] RBAC: Authorizing `identity:get_project(project_id=a54bad337df54d0a81bdc2fc94a6c3e5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:21.544786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24884b2e-78b4-4c64-b929-11372d8ee93e None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:21.546186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d92b60-2f4a-4f41-8613-8b709a077913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:21.546353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d92b60-2f4a-4f41-8613-8b709a077913 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:21.546456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 541/1080] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:13:21 2022] GET /identity/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:21.546876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d92b60-2f4a-4f41-8613-8b709a077913 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:21.547157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d92b60-2f4a-4f41-8613-8b709a077913 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:21.552692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24884b2e-78b4-4c64-b929-11372d8ee93e None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:21.554822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d92b60-2f4a-4f41-8613-8b709a077913 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:21.556269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d92b60-2f4a-4f41-8613-8b709a077913 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-16d92b60-2f4a-4f41-8613-8b709a077913 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:13:21.568133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:21.569447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:21.570401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:13:21.570401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:13:21.570401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:13:21.570401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:13:21.570401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:13:21.570401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:13:21.570401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:13:21.570401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:21.570401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 540/1081] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:13:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:21.575247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d92b60-2f4a-4f41-8613-8b709a077913 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:21.608810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:21.633924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:21.663094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24884b2e-78b4-4c64-b929-11372d8ee93e None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:21.700081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23795b9-f6eb-4e04-b825-a49643e035fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:21.700663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d92b60-2f4a-4f41-8613-8b709a077913 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:21.700779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23795b9-f6eb-4e04-b825-a49643e035fa None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:21.701032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23795b9-f6eb-4e04-b825-a49643e035fa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:21.701311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23795b9-f6eb-4e04-b825-a49643e035fa None admin] PATH_INFO: `/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5/users/9f52e136264d473fac028910e57f94f1/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:21.703382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23795b9-f6eb-4e04-b825-a49643e035fa None admin] RBAC: Authorizing `identity:create_grant(project_id=a54bad337df54d0a81bdc2fc94a6c3e5, user_id=9f52e136264d473fac028910e57f94f1, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:21.704855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23795b9-f6eb-4e04-b825-a49643e035fa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:21.719276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 542/1082] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:13:21 2022] PUT /identity/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5/users/9f52e136264d473fac028910e57f94f1/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:13:21.744283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c54bc-b732-49db-83cb-e95131b6e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:21.744941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c54bc-b732-49db-83cb-e95131b6e2fb None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:21.745222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c54bc-b732-49db-83cb-e95131b6e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:21.745497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c54bc-b732-49db-83cb-e95131b6e2fb None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:21.746351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c54bc-b732-49db-83cb-e95131b6e2fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:21.746659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c54bc-b732-49db-83cb-e95131b6e2fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:21.748211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7c54bc-b732-49db-83cb-e95131b6e2fb None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:21.763275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 541/1083] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:13:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:21.769703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23795b9-f6eb-4e04-b825-a49643e035fa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:21.896187 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23795b9-f6eb-4e04-b825-a49643e035fa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:21.912542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:21.940658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39bd84-fa34-4092-b2fb-7d9ec0ec9a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:21.941415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39bd84-fa34-4092-b2fb-7d9ec0ec9a16 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:21.941699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39bd84-fa34-4092-b2fb-7d9ec0ec9a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:21.942013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39bd84-fa34-4092-b2fb-7d9ec0ec9a16 None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/6d50427014064f099af87932c77a849a/roles/06dfeb3edc3b4716b86a7f7be42d71b1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:21.944280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39bd84-fa34-4092-b2fb-7d9ec0ec9a16 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=6d50427014064f099af87932c77a849a, role_id=06dfeb3edc3b4716b86a7f7be42d71b1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:21.945771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39bd84-fa34-4092-b2fb-7d9ec0ec9a16 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:21.961638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 543/1084] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:13:21 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/6d50427014064f099af87932c77a849a/roles/06dfeb3edc3b4716b86a7f7be42d71b1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:13:23.958882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b031fc1-c51e-4138-8362-f3243784ded3 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:23.959449 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b031fc1-c51e-4138-8362-f3243784ded3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:23.959786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b031fc1-c51e-4138-8362-f3243784ded3 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:23.962239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 542/1085] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:23.967938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43bea75-06e0-4e0c-9b34-a5be57b58ddb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:23.968756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43bea75-06e0-4e0c-9b34-a5be57b58ddb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:23.968756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43bea75-06e0-4e0c-9b34-a5be57b58ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:23.989077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d43bea75-06e0-4e0c-9b34-a5be57b58ddb None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:23.990322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d43bea75-06e0-4e0c-9b34-a5be57b58ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:24.096170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 544/1086] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:13:24.119464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b031fc1-c51e-4138-8362-f3243784ded3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:24.273832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b031fc1-c51e-4138-8362-f3243784ded3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:24.287257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:24.316231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf86a05-d769-43b0-8810-f4bec4eb90d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:24.316977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf86a05-d769-43b0-8810-f4bec4eb90d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:24.317322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf86a05-d769-43b0-8810-f4bec4eb90d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:24.317661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf86a05-d769-43b0-8810-f4bec4eb90d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:24.330157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf86a05-d769-43b0-8810-f4bec4eb90d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-cbf86a05-d769-43b0-8810-f4bec4eb90d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:13:24.335554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:24.346179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:24.348376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 543/1087] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Jul 26 07:13:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:24.350101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43bea75-06e0-4e0c-9b34-a5be57b58ddb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:24.390202 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdf2e3-7972-4159-baa5-db45a7bec853 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:24.390696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdf2e3-7972-4159-baa5-db45a7bec853 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:24.391148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afdf2e3-7972-4159-baa5-db45a7bec853 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:24.393211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 544/1088] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:13:24.402305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13836a2c-53ca-466f-8003-cee7fe310dd3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:24.402680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13836a2c-53ca-466f-8003-cee7fe310dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:24.402992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13836a2c-53ca-466f-8003-cee7fe310dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:24.424864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-13836a2c-53ca-466f-8003-cee7fe310dd3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:24.426344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-13836a2c-53ca-466f-8003-cee7fe310dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:24.479546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:24.483615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43bea75-06e0-4e0c-9b34-a5be57b58ddb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:24.521787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 545/1089] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:13:24.528675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c97a-cf52-467f-940c-cdda9e2f0632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:24.529364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c97a-cf52-467f-940c-cdda9e2f0632 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:24.529773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c97a-cf52-467f-940c-cdda9e2f0632 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:24.530144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9c97a-cf52-467f-940c-cdda9e2f0632 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:24.530984 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9c97a-cf52-467f-940c-cdda9e2f0632 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:24.531383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9c97a-cf52-467f-940c-cdda9e2f0632 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:24.532944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e9c97a-cf52-467f-940c-cdda9e2f0632 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:24.542490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13836a2c-53ca-466f-8003-cee7fe310dd3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:24.543880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 545/1090] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jul 26 07:13:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:24.550414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c97a-cf52-467f-940c-cdda9e2f0632 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:24.674241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9c97a-cf52-467f-940c-cdda9e2f0632 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:24.684892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:24.690422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13836a2c-53ca-466f-8003-cee7fe310dd3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:24.710867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e41508-5f74-4446-810b-f69ec9b8a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:24.711653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e41508-5f74-4446-810b-f69ec9b8a6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:24.711992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e41508-5f74-4446-810b-f69ec9b8a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:24.712318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e41508-5f74-4446-810b-f69ec9b8a6c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:24.731910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37080b-906c-45fa-baad-0737be7b8bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:24.732668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37080b-906c-45fa-baad-0737be7b8bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:24.732949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37080b-906c-45fa-baad-0737be7b8bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:24.733216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37080b-906c-45fa-baad-0737be7b8bb8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:24.735095 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e41508-5f74-4446-810b-f69ec9b8a6c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:24.736569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e41508-5f74-4446-810b-f69ec9b8a6c0 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:24.744991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37080b-906c-45fa-baad-0737be7b8bb8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7b37080b-906c-45fa-baad-0737be7b8bb8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:13:24.749952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:24.751185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:24.752398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 546/1091] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:13:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d5e41508-5f74-4446-810b-f69ec9b8a6c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:24.753258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:24.755732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:24.757508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:13:24.757508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:24.757508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:24.757508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:24.757508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 546/1092] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:24.757977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37080b-906c-45fa-baad-0737be7b8bb8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:24.759293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e41508-5f74-4446-810b-f69ec9b8a6c0 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:24.775807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:24.872137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e41508-5f74-4446-810b-f69ec9b8a6c0 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:24.874623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37080b-906c-45fa-baad-0737be7b8bb8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:24.907841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5366c1-65e2-4b7e-9639-cea0c52ef503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:24.908469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5366c1-65e2-4b7e-9639-cea0c52ef503 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:24.908745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5366c1-65e2-4b7e-9639-cea0c52ef503 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:24.909014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5366c1-65e2-4b7e-9639-cea0c52ef503 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:24.909703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5366c1-65e2-4b7e-9639-cea0c52ef503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:24.910118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5366c1-65e2-4b7e-9639-cea0c52ef503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:24.911914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5366c1-65e2-4b7e-9639-cea0c52ef503 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:24.914965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08834159-13d9-4565-a6bc-80ddbcbfb329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:24.915671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08834159-13d9-4565-a6bc-80ddbcbfb329 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:24.916035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08834159-13d9-4565-a6bc-80ddbcbfb329 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:24.916479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08834159-13d9-4565-a6bc-80ddbcbfb329 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:24.917383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08834159-13d9-4565-a6bc-80ddbcbfb329 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:24.917715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08834159-13d9-4565-a6bc-80ddbcbfb329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:24.919480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08834159-13d9-4565-a6bc-80ddbcbfb329 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:24.924459 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 547/1093] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:24.930787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5366c1-65e2-4b7e-9639-cea0c52ef503 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:24.936640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 547/1094] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:13:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:24.944375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08834159-13d9-4565-a6bc-80ddbcbfb329 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:24.980506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:25.028930 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:25.058234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5366c1-65e2-4b7e-9639-cea0c52ef503 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:25.083809 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08834159-13d9-4565-a6bc-80ddbcbfb329 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:25.098524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc96de5-594d-4986-a22e-d4f068457f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:25.099625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc96de5-594d-4986-a22e-d4f068457f1a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:25.100067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc96de5-594d-4986-a22e-d4f068457f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:25.100512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc96de5-594d-4986-a22e-d4f068457f1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:25.110809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc96de5-594d-4986-a22e-d4f068457f1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:25.112431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc96de5-594d-4986-a22e-d4f068457f1a None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:25.119831 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dabc3d-b73b-40fb-9cdc-6eca5fa1d607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:25.120491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dabc3d-b73b-40fb-9cdc-6eca5fa1d607 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:25.120773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dabc3d-b73b-40fb-9cdc-6eca5fa1d607 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:25.121056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dabc3d-b73b-40fb-9cdc-6eca5fa1d607 None admin] PATH_INFO: `/v3/users/9f52e136264d473fac028910e57f94f1` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:25.122384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dabc3d-b73b-40fb-9cdc-6eca5fa1d607 None admin] RBAC: Authorizing `identity:get_user(user_id=9f52e136264d473fac028910e57f94f1)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:25.123727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dabc3d-b73b-40fb-9cdc-6eca5fa1d607 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:25.125523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 548/1095] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:24 2022] GET /identity/v3/users/9f52e136264d473fac028910e57f94f1 => generated 285 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-4fc96de5-594d-4986-a22e-d4f068457f1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:13:25.126677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:25.127653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:25.128616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 548/1096] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Jul 26 07:13:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:25.131278 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dabc3d-b73b-40fb-9cdc-6eca5fa1d607 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:25.134508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc96de5-594d-4986-a22e-d4f068457f1a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:25.235751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:25.247011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dabc3d-b73b-40fb-9cdc-6eca5fa1d607 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:25.253135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc96de5-594d-4986-a22e-d4f068457f1a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:25.285987 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892e383-6e54-4b18-a4e6-f7398626211d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:25.286620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e383-6e54-4b18-a4e6-f7398626211d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:25.286916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e383-6e54-4b18-a4e6-f7398626211d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:25.287183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e383-6e54-4b18-a4e6-f7398626211d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:25.297948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7fc0-7b70-4b37-93dc-c10c0101cebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:25.298530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7fc0-7b70-4b37-93dc-c10c0101cebb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:25.298805 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7fc0-7b70-4b37-93dc-c10c0101cebb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:25.299075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f7fc0-7b70-4b37-93dc-c10c0101cebb None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:25.299874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f7fc0-7b70-4b37-93dc-c10c0101cebb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:25.300178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f7fc0-7b70-4b37-93dc-c10c0101cebb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:25.302059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f7fc0-7b70-4b37-93dc-c10c0101cebb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:25.304004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:25.323372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 549/1097] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Jul 26 07:13:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:25.332734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7fc0-7b70-4b37-93dc-c10c0101cebb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:25.384863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892e383-6e54-4b18-a4e6-f7398626211d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:25.387347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892e383-6e54-4b18-a4e6-f7398626211d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:25.453155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f7fc0-7b70-4b37-93dc-c10c0101cebb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:25.493274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ecffc-4abf-430a-9a35-ebeffb374c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:25.493949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ecffc-4abf-430a-9a35-ebeffb374c73 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:25.494228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ecffc-4abf-430a-9a35-ebeffb374c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:25.494481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ecffc-4abf-430a-9a35-ebeffb374c73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:25.503801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:25.521406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 549/1098] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 07:13:25.527556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892e383-6e54-4b18-a4e6-f7398626211d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:25.555662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:25.566050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ecffc-4abf-430a-9a35-ebeffb374c73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:25.567968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348ecffc-4abf-430a-9a35-ebeffb374c73 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:25.640933 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892e383-6e54-4b18-a4e6-f7398626211d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:25.680755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 550/1099] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 07:13:25.680996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4e9d8-6f57-494a-a072-f5351a805794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:25.680996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4e9d8-6f57-494a-a072-f5351a805794 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:25.680996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4e9d8-6f57-494a-a072-f5351a805794 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:25.680996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4e9d8-6f57-494a-a072-f5351a805794 None admin] PATH_INFO: `/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:25.681350 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4e9d8-6f57-494a-a072-f5351a805794 None admin] RBAC: Authorizing `identity:get_project(project_id=a54bad337df54d0a81bdc2fc94a6c3e5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:25.682010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ecffc-4abf-430a-9a35-ebeffb374c73 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:25.682407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4e9d8-6f57-494a-a072-f5351a805794 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:25.684322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 550/1100] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:13:25 2022] GET /identity/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:25.690378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4e9d8-6f57-494a-a072-f5351a805794 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:25.759063 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:25.788249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:25.797667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348ecffc-4abf-430a-9a35-ebeffb374c73 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:25.816073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4e9d8-6f57-494a-a072-f5351a805794 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:25.834166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095707ab-a117-4321-b3d5-c5ae03637c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:25.834834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095707ab-a117-4321-b3d5-c5ae03637c76 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:25.835109 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095707ab-a117-4321-b3d5-c5ae03637c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:25.835382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095707ab-a117-4321-b3d5-c5ae03637c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:25.846980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095707ab-a117-4321-b3d5-c5ae03637c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:25.848517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095707ab-a117-4321-b3d5-c5ae03637c76 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:25.859605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f1e71-5177-4476-9d88-2adbef3b6715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:25.860263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f1e71-5177-4476-9d88-2adbef3b6715 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:25.860541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f1e71-5177-4476-9d88-2adbef3b6715 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:25.860814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f1e71-5177-4476-9d88-2adbef3b6715 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:25.861613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f1e71-5177-4476-9d88-2adbef3b6715 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=a54bad337df54d0a81bdc2fc94a6c3e5, user.id=9f52e136264d473fac028910e57f94f1) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:25.861956 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f1e71-5177-4476-9d88-2adbef3b6715 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:25.863438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f1e71-5177-4476-9d88-2adbef3b6715 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:25.877326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 551/1101] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:13:25 2022] GET /identity/v3/role_assignments?user.id=9f52e136264d473fac028910e57f94f1&scope.project.id=a54bad337df54d0a81bdc2fc94a6c3e5&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-095707ab-a117-4321-b3d5-c5ae03637c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:25.877568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:25.878803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:25.879907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:13:25.879907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:25.879907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:13:25.879907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:25.879907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 551/1102] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:25.883749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f1e71-5177-4476-9d88-2adbef3b6715 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:26.001349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f1e71-5177-4476-9d88-2adbef3b6715 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:26.027882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:26.048799 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3709e-6e4f-4f0d-9259-f1d8444c3251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:26.049431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3709e-6e4f-4f0d-9259-f1d8444c3251 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:26.049708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3709e-6e4f-4f0d-9259-f1d8444c3251 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:26.050034 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3709e-6e4f-4f0d-9259-f1d8444c3251 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:26.050825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3709e-6e4f-4f0d-9259-f1d8444c3251 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:26.051128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3709e-6e4f-4f0d-9259-f1d8444c3251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:26.052499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3709e-6e4f-4f0d-9259-f1d8444c3251 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:26.069792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 552/1103] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:26.075848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095707ab-a117-4321-b3d5-c5ae03637c76 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:26.191066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095707ab-a117-4321-b3d5-c5ae03637c76 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:26.227424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b14830-6e68-4be1-85eb-aacfcd63c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:26.228075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b14830-6e68-4be1-85eb-aacfcd63c34d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:26.228339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b14830-6e68-4be1-85eb-aacfcd63c34d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:26.228615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b14830-6e68-4be1-85eb-aacfcd63c34d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:26.236596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b14830-6e68-4be1-85eb-aacfcd63c34d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:26.238146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b14830-6e68-4be1-85eb-aacfcd63c34d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-47b14830-6e68-4be1-85eb-aacfcd63c34d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:13:26.250170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:26.251215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:26.252677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:13:26.252677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:13:26.252677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:13:26.252677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:13:26.252677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:13:26.252677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:13:26.252677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:13:26.252677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:26.252677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 552/1104] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:26.257477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3709e-6e4f-4f0d-9259-f1d8444c3251 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:26.388381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3709e-6e4f-4f0d-9259-f1d8444c3251 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:26.430190 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294498fa-68c9-48db-afba-4e20d1561876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:26.430838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294498fa-68c9-48db-afba-4e20d1561876 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:26.431117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294498fa-68c9-48db-afba-4e20d1561876 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:26.431390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294498fa-68c9-48db-afba-4e20d1561876 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:26.432164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294498fa-68c9-48db-afba-4e20d1561876 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:26.432471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294498fa-68c9-48db-afba-4e20d1561876 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:26.433989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294498fa-68c9-48db-afba-4e20d1561876 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:26.448738 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 553/1105] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:26.455781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b14830-6e68-4be1-85eb-aacfcd63c34d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:26.462402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:26.580671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b14830-6e68-4be1-85eb-aacfcd63c34d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:26.624421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058a725d-85c0-4e08-b553-06fed8d227c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:26.625078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a725d-85c0-4e08-b553-06fed8d227c7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:26.625361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a725d-85c0-4e08-b553-06fed8d227c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:26.625637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058a725d-85c0-4e08-b553-06fed8d227c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:26.626512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a725d-85c0-4e08-b553-06fed8d227c7 None admin] RBAC: Adding query filter params (role.id=06dfeb3edc3b4716b86a7f7be42d71b1, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=6d50427014064f099af87932c77a849a) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:26.626823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a725d-85c0-4e08-b553-06fed8d227c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:26.628351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058a725d-85c0-4e08-b553-06fed8d227c7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:26.635805 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 553/1106] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:13:26 2022] GET /identity/v3/role_assignments?user.id=6d50427014064f099af87932c77a849a&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=06dfeb3edc3b4716b86a7f7be42d71b1 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:27.968500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d8511-13af-4860-8506-b9301fd13f7a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:27.969127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d8511-13af-4860-8506-b9301fd13f7a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:27.969127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d8511-13af-4860-8506-b9301fd13f7a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:27.970478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 554/1107] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:27.977158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d4353-f5be-4b40-b65c-a612ceb0dfc1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:27.977640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d4353-f5be-4b40-b65c-a612ceb0dfc1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:27.978098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d4353-f5be-4b40-b65c-a612ceb0dfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:28.008284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-172d4353-f5be-4b40-b65c-a612ceb0dfc1 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:28.010243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-172d4353-f5be-4b40-b65c-a612ceb0dfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:28.114829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:28.124269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 554/1108] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:28.148790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d8511-13af-4860-8506-b9301fd13f7a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:28.213468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:28.307514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d8511-13af-4860-8506-b9301fd13f7a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:28.367895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c721e5fd-8614-47e7-b53c-b7a79c46f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:28.368950 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721e5fd-8614-47e7-b53c-b7a79c46f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:28.369422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721e5fd-8614-47e7-b53c-b7a79c46f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:28.370017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c721e5fd-8614-47e7-b53c-b7a79c46f9d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:28.371944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721e5fd-8614-47e7-b53c-b7a79c46f9d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:28.374565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c721e5fd-8614-47e7-b53c-b7a79c46f9d2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:28.377439 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 555/1109] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:28.386422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d4353-f5be-4b40-b65c-a612ceb0dfc1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:28.494424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d4353-f5be-4b40-b65c-a612ceb0dfc1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:28.529457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c8e28-e49a-4196-a5f0-da8c605daeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:28.530218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8e28-e49a-4196-a5f0-da8c605daeeb None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:28.530543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8e28-e49a-4196-a5f0-da8c605daeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:28.530860 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8e28-e49a-4196-a5f0-da8c605daeeb None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:28.531602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c8e28-e49a-4196-a5f0-da8c605daeeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:28.533108 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c8e28-e49a-4196-a5f0-da8c605daeeb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:28.556476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 555/1110] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:13:28 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:13:30.607946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bcbe2-7add-4479-8b58-e1b9b67829f6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:30.608509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bcbe2-7add-4479-8b58-e1b9b67829f6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:30.608509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bcbe2-7add-4479-8b58-e1b9b67829f6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:30.610295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 556/1111] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:30.618645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269892b-d8bf-4538-b0ba-32c3d33480c0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:30.619245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269892b-d8bf-4538-b0ba-32c3d33480c0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:30.619245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7269892b-d8bf-4538-b0ba-32c3d33480c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:30.638643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7269892b-d8bf-4538-b0ba-32c3d33480c0 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:30.639866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7269892b-d8bf-4538-b0ba-32c3d33480c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:30.692402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:30.713027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 556/1112] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:13:30.737666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bcbe2-7add-4479-8b58-e1b9b67829f6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:30.850415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bcbe2-7add-4479-8b58-e1b9b67829f6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:30.863449 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:30.902141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725924b1-b6c7-4ad0-bf80-9df112d10b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:30.903248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725924b1-b6c7-4ad0-bf80-9df112d10b64 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:30.903712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725924b1-b6c7-4ad0-bf80-9df112d10b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:30.904175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725924b1-b6c7-4ad0-bf80-9df112d10b64 None admin] PATH_INFO: `/v3/roles/5e0c08646046426385776839f1f168cd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:30.907502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725924b1-b6c7-4ad0-bf80-9df112d10b64 None admin] RBAC: Authorizing `identity:get_role(role_id=5e0c08646046426385776839f1f168cd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:30.910127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725924b1-b6c7-4ad0-bf80-9df112d10b64 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:30.912544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 557/1113] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:30 2022] GET /identity/v3/roles/5e0c08646046426385776839f1f168cd => generated 223 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:30.919054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269892b-d8bf-4538-b0ba-32c3d33480c0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:31.029355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7269892b-d8bf-4538-b0ba-32c3d33480c0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:31.065355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c82715-4a7e-4834-9364-502f07897db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:31.066045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c82715-4a7e-4834-9364-502f07897db3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:31.066371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c82715-4a7e-4834-9364-502f07897db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:31.066645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c82715-4a7e-4834-9364-502f07897db3 None admin] PATH_INFO: `/v3/users/f0b5ac2af16a4a5b89749761331426c2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:31.081544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c82715-4a7e-4834-9364-502f07897db3 None admin] RBAC: Authorizing `identity:get_user(user_id=f0b5ac2af16a4a5b89749761331426c2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:31.083255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c82715-4a7e-4834-9364-502f07897db3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:31.085013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 557/1114] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:30 2022] GET /identity/v3/users/f0b5ac2af16a4a5b89749761331426c2 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:31.091198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725924b1-b6c7-4ad0-bf80-9df112d10b64 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:31.226057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725924b1-b6c7-4ad0-bf80-9df112d10b64 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:31.261659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cce17f-9e64-4b44-9ead-bcf414f7f6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:31.262323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cce17f-9e64-4b44-9ead-bcf414f7f6a2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:31.262605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cce17f-9e64-4b44-9ead-bcf414f7f6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:31.262880 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cce17f-9e64-4b44-9ead-bcf414f7f6a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:31.320627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:31.342979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cce17f-9e64-4b44-9ead-bcf414f7f6a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:31.344440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cce17f-9e64-4b44-9ead-bcf414f7f6a2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:31.451435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 558/1115] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 07:13:31.457692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c82715-4a7e-4834-9364-502f07897db3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:31.510851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:31.584340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c82715-4a7e-4834-9364-502f07897db3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:31.624910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d323ba-e3c3-4a5e-94a7-29dc599673c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:31.625645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d323ba-e3c3-4a5e-94a7-29dc599673c1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:31.625854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d323ba-e3c3-4a5e-94a7-29dc599673c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:31.626141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d323ba-e3c3-4a5e-94a7-29dc599673c1 None admin] PATH_INFO: `/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:31.627437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d323ba-e3c3-4a5e-94a7-29dc599673c1 None admin] RBAC: Authorizing `identity:get_project(project_id=a54bad337df54d0a81bdc2fc94a6c3e5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:31.628981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d323ba-e3c3-4a5e-94a7-29dc599673c1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:31.630825 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 558/1116] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:13:31 2022] GET /identity/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5 => generated 311 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:31.637510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cce17f-9e64-4b44-9ead-bcf414f7f6a2 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:31.721854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:31.745427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cce17f-9e64-4b44-9ead-bcf414f7f6a2 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:31.785259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27402b1d-e163-48c0-b431-128bd6c99cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:31.786303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27402b1d-e163-48c0-b431-128bd6c99cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:31.786770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27402b1d-e163-48c0-b431-128bd6c99cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:31.787246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27402b1d-e163-48c0-b431-128bd6c99cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:31.788731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27402b1d-e163-48c0-b431-128bd6c99cd9 None admin] RBAC: Adding query filter params (role.id=5e0c08646046426385776839f1f168cd, scope.project.id=a54bad337df54d0a81bdc2fc94a6c3e5, user.id=f0b5ac2af16a4a5b89749761331426c2) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:31.789194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27402b1d-e163-48c0-b431-128bd6c99cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:31.791385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27402b1d-e163-48c0-b431-128bd6c99cd9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:31.799652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 559/1117] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:13:31 2022] GET /identity/v3/role_assignments?user.id=f0b5ac2af16a4a5b89749761331426c2&scope.project.id=a54bad337df54d0a81bdc2fc94a6c3e5&role.id=5e0c08646046426385776839f1f168cd => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:33.827620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c891c-e7e2-48a7-9cb3-4f9f687905ce None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:33.828197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c891c-e7e2-48a7-9cb3-4f9f687905ce None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:33.828197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c891c-e7e2-48a7-9cb3-4f9f687905ce None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:33.829667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 559/1118] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:33.835799 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b126-4e21-438f-be0e-9c413d738b05 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:33.836293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b126-4e21-438f-be0e-9c413d738b05 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:33.836412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b126-4e21-438f-be0e-9c413d738b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:33.856063 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-30f6b126-4e21-438f-be0e-9c413d738b05 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:33.857382 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-30f6b126-4e21-438f-be0e-9c413d738b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:33.932617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 560/1119] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:33.955525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c891c-e7e2-48a7-9cb3-4f9f687905ce None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:34.066194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c891c-e7e2-48a7-9cb3-4f9f687905ce None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:34.078905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:34.106787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385983e-ee0d-47a5-9c4a-e55909f90bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:34.107448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385983e-ee0d-47a5-9c4a-e55909f90bef None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:34.107728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385983e-ee0d-47a5-9c4a-e55909f90bef None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:34.107999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385983e-ee0d-47a5-9c4a-e55909f90bef None admin] PATH_INFO: `/v3/roles/5e0c08646046426385776839f1f168cd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:34.109867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385983e-ee0d-47a5-9c4a-e55909f90bef None admin] RBAC: Authorizing `identity:get_role(role_id=5e0c08646046426385776839f1f168cd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:34.111249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385983e-ee0d-47a5-9c4a-e55909f90bef None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:34.112661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 560/1120] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:33 2022] GET /identity/v3/roles/5e0c08646046426385776839f1f168cd => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:34.121612 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6b126-4e21-438f-be0e-9c413d738b05 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:34.198623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:34.233649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6b126-4e21-438f-be0e-9c413d738b05 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:34.287108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d95550-e8da-4efe-9780-127a8c08b851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:34.287765 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d95550-e8da-4efe-9780-127a8c08b851 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:34.288048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d95550-e8da-4efe-9780-127a8c08b851 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:34.288324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d95550-e8da-4efe-9780-127a8c08b851 None admin] PATH_INFO: `/v3/users/f0b5ac2af16a4a5b89749761331426c2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:34.289571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d95550-e8da-4efe-9780-127a8c08b851 None admin] RBAC: Authorizing `identity:get_user(user_id=f0b5ac2af16a4a5b89749761331426c2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:34.291090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d95550-e8da-4efe-9780-127a8c08b851 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:34.292786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 561/1121] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:34 2022] GET /identity/v3/users/f0b5ac2af16a4a5b89749761331426c2 => generated 286 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:34.299232 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385983e-ee0d-47a5-9c4a-e55909f90bef None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:34.408671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385983e-ee0d-47a5-9c4a-e55909f90bef None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:34.448712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf043193-aff5-4cfa-9d77-e1a416e61278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:34.449380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf043193-aff5-4cfa-9d77-e1a416e61278 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:34.449662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf043193-aff5-4cfa-9d77-e1a416e61278 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:34.449964 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf043193-aff5-4cfa-9d77-e1a416e61278 None admin] PATH_INFO: `/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:34.451165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf043193-aff5-4cfa-9d77-e1a416e61278 None admin] RBAC: Authorizing `identity:get_project(project_id=a54bad337df54d0a81bdc2fc94a6c3e5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:34.452616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf043193-aff5-4cfa-9d77-e1a416e61278 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:34.454352 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 561/1122] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:13:34 2022] GET /identity/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:34.464835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d95550-e8da-4efe-9780-127a8c08b851 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:34.617700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d95550-e8da-4efe-9780-127a8c08b851 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:34.656262 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9922459f-9944-435d-9758-24ca70cb08c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:34.656951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922459f-9944-435d-9758-24ca70cb08c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:34.657232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922459f-9944-435d-9758-24ca70cb08c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:34.657504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9922459f-9944-435d-9758-24ca70cb08c4 None admin] PATH_INFO: `/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5/users/f0b5ac2af16a4a5b89749761331426c2/roles/5e0c08646046426385776839f1f168cd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:34.659969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922459f-9944-435d-9758-24ca70cb08c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a54bad337df54d0a81bdc2fc94a6c3e5, user_id=f0b5ac2af16a4a5b89749761331426c2, role_id=5e0c08646046426385776839f1f168cd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:34.661369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9922459f-9944-435d-9758-24ca70cb08c4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:34.672539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:34.677866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 562/1123] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:13:34 2022] PUT /identity/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5/users/f0b5ac2af16a4a5b89749761331426c2/roles/5e0c08646046426385776839f1f168cd => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:13:36.703943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db73cc8-66dc-4aff-853b-813297e1aff8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:36.704499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db73cc8-66dc-4aff-853b-813297e1aff8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:36.704499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db73cc8-66dc-4aff-853b-813297e1aff8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:36.705762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 562/1124] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:36.711642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd181-c106-4206-91bb-411d082b62cc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:36.712231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd181-c106-4206-91bb-411d082b62cc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:36.712231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fd181-c106-4206-91bb-411d082b62cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:36.731663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d63fd181-c106-4206-91bb-411d082b62cc None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:36.732898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d63fd181-c106-4206-91bb-411d082b62cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:36.807101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 563/1125] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:36.831006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db73cc8-66dc-4aff-853b-813297e1aff8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:36.884500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:36.943538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db73cc8-66dc-4aff-853b-813297e1aff8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:36.981522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94225fe0-bd16-43a8-9043-3fae8c34f110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:36.982115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94225fe0-bd16-43a8-9043-3fae8c34f110 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:36.982424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94225fe0-bd16-43a8-9043-3fae8c34f110 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:36.982694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94225fe0-bd16-43a8-9043-3fae8c34f110 None admin] PATH_INFO: `/v3/roles/5e0c08646046426385776839f1f168cd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:36.984192 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94225fe0-bd16-43a8-9043-3fae8c34f110 None admin] RBAC: Authorizing `identity:get_role(role_id=5e0c08646046426385776839f1f168cd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:36.986089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94225fe0-bd16-43a8-9043-3fae8c34f110 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:36.987477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 563/1126] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:36 2022] GET /identity/v3/roles/5e0c08646046426385776839f1f168cd => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:36.993960 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fd181-c106-4206-91bb-411d082b62cc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:37.112428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fd181-c106-4206-91bb-411d082b62cc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:37.143378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:37.150330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2bd32-cddc-4fa8-8fe5-0f326c804880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:37.151050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2bd32-cddc-4fa8-8fe5-0f326c804880 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:37.151378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2bd32-cddc-4fa8-8fe5-0f326c804880 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:37.151699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2bd32-cddc-4fa8-8fe5-0f326c804880 None admin] PATH_INFO: `/v3/users/f0b5ac2af16a4a5b89749761331426c2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:37.152958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2bd32-cddc-4fa8-8fe5-0f326c804880 None admin] RBAC: Authorizing `identity:get_user(user_id=f0b5ac2af16a4a5b89749761331426c2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:37.154666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2bd32-cddc-4fa8-8fe5-0f326c804880 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:37.156414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 564/1127] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:36 2022] GET /identity/v3/users/f0b5ac2af16a4a5b89749761331426c2 => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:37.162740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94225fe0-bd16-43a8-9043-3fae8c34f110 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:37.267209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94225fe0-bd16-43a8-9043-3fae8c34f110 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:37.282485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:37.304226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a23ef4-5e88-4120-9757-c4e9c7e0daeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:37.304902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a23ef4-5e88-4120-9757-c4e9c7e0daeb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:37.305227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a23ef4-5e88-4120-9757-c4e9c7e0daeb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:37.305542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a23ef4-5e88-4120-9757-c4e9c7e0daeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:37.376317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a23ef4-5e88-4120-9757-c4e9c7e0daeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:37.377936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a23ef4-5e88-4120-9757-c4e9c7e0daeb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:37.484964 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 564/1128] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 07:13:37.491467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2bd32-cddc-4fa8-8fe5-0f326c804880 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:37.626375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2bd32-cddc-4fa8-8fe5-0f326c804880 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:37.685764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1d705-8a38-487f-b3f5-6fabbcebb621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:37.687101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d705-8a38-487f-b3f5-6fabbcebb621 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:37.687576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d705-8a38-487f-b3f5-6fabbcebb621 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:37.688055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1d705-8a38-487f-b3f5-6fabbcebb621 None admin] PATH_INFO: `/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:37.690080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1d705-8a38-487f-b3f5-6fabbcebb621 None admin] RBAC: Authorizing `identity:get_project(project_id=a54bad337df54d0a81bdc2fc94a6c3e5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:37.692339 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1d705-8a38-487f-b3f5-6fabbcebb621 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:37.695423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 565/1129] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:13:37 2022] GET /identity/v3/projects/a54bad337df54d0a81bdc2fc94a6c3e5 => generated 311 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:37.702079 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a23ef4-5e88-4120-9757-c4e9c7e0daeb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:37.719400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:37.816423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a23ef4-5e88-4120-9757-c4e9c7e0daeb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:37.853297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea2ec7-d1c3-4def-8d2e-92c4ec8e0c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:37.853956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea2ec7-d1c3-4def-8d2e-92c4ec8e0c95 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:37.854231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea2ec7-d1c3-4def-8d2e-92c4ec8e0c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:37.854496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea2ec7-d1c3-4def-8d2e-92c4ec8e0c95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:37.855287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea2ec7-d1c3-4def-8d2e-92c4ec8e0c95 None admin] RBAC: Adding query filter params (role.id=5e0c08646046426385776839f1f168cd, scope.project.id=a54bad337df54d0a81bdc2fc94a6c3e5, user.id=f0b5ac2af16a4a5b89749761331426c2) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:37.855590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea2ec7-d1c3-4def-8d2e-92c4ec8e0c95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:37.857034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea2ec7-d1c3-4def-8d2e-92c4ec8e0c95 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:37.863959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 565/1130] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:13:37 2022] GET /identity/v3/role_assignments?user.id=f0b5ac2af16a4a5b89749761331426c2&scope.project.id=a54bad337df54d0a81bdc2fc94a6c3e5&role.id=5e0c08646046426385776839f1f168cd => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:39.907290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ef87e-fae1-4ddd-a2ca-54d6a9cf40a2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:39.907919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ef87e-fae1-4ddd-a2ca-54d6a9cf40a2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:39.907919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ef87e-fae1-4ddd-a2ca-54d6a9cf40a2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:39.909195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 566/1131] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:39.914804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108a162-2a5a-4451-aac3-91d0b6409676 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:39.915290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108a162-2a5a-4451-aac3-91d0b6409676 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:39.915388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108a162-2a5a-4451-aac3-91d0b6409676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:39.934586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9108a162-2a5a-4451-aac3-91d0b6409676 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:39.935813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9108a162-2a5a-4451-aac3-91d0b6409676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:40.010825 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 566/1132] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:40.034748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ef87e-fae1-4ddd-a2ca-54d6a9cf40a2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:40.169502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:40.212527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ef87e-fae1-4ddd-a2ca-54d6a9cf40a2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:40.250580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ca9449-8209-4680-9c56-712a985de7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:40.251231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca9449-8209-4680-9c56-712a985de7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:40.251514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca9449-8209-4680-9c56-712a985de7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:40.251788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ca9449-8209-4680-9c56-712a985de7a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:40.252974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca9449-8209-4680-9c56-712a985de7a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:40.254507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ca9449-8209-4680-9c56-712a985de7a3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:40.256137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 567/1133] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:40.263394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108a162-2a5a-4451-aac3-91d0b6409676 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:40.277690 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:40.377401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108a162-2a5a-4451-aac3-91d0b6409676 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:40.418322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ead7cf-8c25-424f-96a9-7eb6d83cda19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:40.418983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ead7cf-8c25-424f-96a9-7eb6d83cda19 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:40.419264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ead7cf-8c25-424f-96a9-7eb6d83cda19 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:40.419591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ead7cf-8c25-424f-96a9-7eb6d83cda19 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:40.420269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ead7cf-8c25-424f-96a9-7eb6d83cda19 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:40.421821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ead7cf-8c25-424f-96a9-7eb6d83cda19 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:40.459879 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 567/1134] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:13:40 2022] POST /identity/v3/projects => generated 311 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:13:43.529242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596c690-b25f-4fc3-b45f-b98cfd01b9e7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:43.529822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596c690-b25f-4fc3-b45f-b98cfd01b9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:43.529942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596c690-b25f-4fc3-b45f-b98cfd01b9e7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:43.531345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 568/1135] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:43.537325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fef79-f024-451a-a95b-694473b6991d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:43.537940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fef79-f024-451a-a95b-694473b6991d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:43.537940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fef79-f024-451a-a95b-694473b6991d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:43.556570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-426fef79-f024-451a-a95b-694473b6991d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:43.557831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-426fef79-f024-451a-a95b-694473b6991d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:43.619366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:43.633602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 568/1136] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:43.658770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596c690-b25f-4fc3-b45f-b98cfd01b9e7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:43.772949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596c690-b25f-4fc3-b45f-b98cfd01b9e7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:43.813504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf2b5e-f995-40f2-ac2d-d452012bce15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:43.814206 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf2b5e-f995-40f2-ac2d-d452012bce15 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:43.814490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf2b5e-f995-40f2-ac2d-d452012bce15 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:43.814767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf2b5e-f995-40f2-ac2d-d452012bce15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:43.816069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf2b5e-f995-40f2-ac2d-d452012bce15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:43.817616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edf2b5e-f995-40f2-ac2d-d452012bce15 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:43.819333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 569/1137] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:13:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:43.825538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fef79-f024-451a-a95b-694473b6991d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:43.982724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fef79-f024-451a-a95b-694473b6991d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:44.043517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049111f-2592-4e3f-b298-f918b2424be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:44.044493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049111f-2592-4e3f-b298-f918b2424be9 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:44.044992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049111f-2592-4e3f-b298-f918b2424be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:44.045472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049111f-2592-4e3f-b298-f918b2424be9 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:44.046772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049111f-2592-4e3f-b298-f918b2424be9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:44.049469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049111f-2592-4e3f-b298-f918b2424be9 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:44.085165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 569/1138] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:13:43 2022] POST /identity/v3/users => generated 286 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:13:46.163639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f772a-4a04-4cb8-8b23-f1755eddffbe None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:46.164488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f772a-4a04-4cb8-8b23-f1755eddffbe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:46.164488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36f772a-4a04-4cb8-8b23-f1755eddffbe None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:46.165572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 570/1139] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:46.176659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df047e3-dc78-4357-bcef-81a794e14cb2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:46.177290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df047e3-dc78-4357-bcef-81a794e14cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:46.177490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df047e3-dc78-4357-bcef-81a794e14cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:46.199352 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5df047e3-dc78-4357-bcef-81a794e14cb2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:46.200654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5df047e3-dc78-4357-bcef-81a794e14cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:46.249807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:46.279933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 570/1140] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:46.303492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f772a-4a04-4cb8-8b23-f1755eddffbe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:46.306671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:46.419884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36f772a-4a04-4cb8-8b23-f1755eddffbe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:46.458719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305dbd4-68ba-4871-a19a-40ca73255a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:46.459392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305dbd4-68ba-4871-a19a-40ca73255a32 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:46.459673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305dbd4-68ba-4871-a19a-40ca73255a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:46.459953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1305dbd4-68ba-4871-a19a-40ca73255a32 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:46.472243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1305dbd4-68ba-4871-a19a-40ca73255a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1305dbd4-68ba-4871-a19a-40ca73255a32 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:13:46.477557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:46.479356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:46.480709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 571/1141] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:13:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:46.486934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df047e3-dc78-4357-bcef-81a794e14cb2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:46.594945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df047e3-dc78-4357-bcef-81a794e14cb2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:46.637559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09078c50-5353-492b-99cf-95a7ca1e9c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:46.638345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09078c50-5353-492b-99cf-95a7ca1e9c13 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:46.638648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09078c50-5353-492b-99cf-95a7ca1e9c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:46.639006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09078c50-5353-492b-99cf-95a7ca1e9c13 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:46.639819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09078c50-5353-492b-99cf-95a7ca1e9c13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:46.640168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09078c50-5353-492b-99cf-95a7ca1e9c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:46.641707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09078c50-5353-492b-99cf-95a7ca1e9c13 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:46.652183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 571/1142] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:13:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:46.658732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305dbd4-68ba-4871-a19a-40ca73255a32 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:46.738309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:46.786169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1305dbd4-68ba-4871-a19a-40ca73255a32 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:46.826271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d95fbe-7506-49ff-91ed-819935b76076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:46.826938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95fbe-7506-49ff-91ed-819935b76076 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:46.827215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95fbe-7506-49ff-91ed-819935b76076 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:46.827489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d95fbe-7506-49ff-91ed-819935b76076 None admin] PATH_INFO: `/v3/users/a321656c417d4a398b229009a2f1e3a6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:46.844656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d95fbe-7506-49ff-91ed-819935b76076 None admin] RBAC: Authorizing `identity:get_user(user_id=a321656c417d4a398b229009a2f1e3a6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:46.846298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d95fbe-7506-49ff-91ed-819935b76076 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:46.848204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 572/1143] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:46 2022] GET /identity/v3/users/a321656c417d4a398b229009a2f1e3a6 => generated 286 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:46.854782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09078c50-5353-492b-99cf-95a7ca1e9c13 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:46.902780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:46.974627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09078c50-5353-492b-99cf-95a7ca1e9c13 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:47.012817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf8562-b280-4e32-b2ee-20985618a895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:47.013452 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf8562-b280-4e32-b2ee-20985618a895 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:47.013779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf8562-b280-4e32-b2ee-20985618a895 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:47.014039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf8562-b280-4e32-b2ee-20985618a895 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:47.091040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf8562-b280-4e32-b2ee-20985618a895 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:47.092599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf8562-b280-4e32-b2ee-20985618a895 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:47.133772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:47.201637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 572/1144] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:13:47.208128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d95fbe-7506-49ff-91ed-819935b76076 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:47.320579 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d95fbe-7506-49ff-91ed-819935b76076 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:47.342901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:47.359238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414c606-ef3f-4908-8f23-cb893e76872a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:47.360323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414c606-ef3f-4908-8f23-cb893e76872a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:47.360648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414c606-ef3f-4908-8f23-cb893e76872a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:47.360972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414c606-ef3f-4908-8f23-cb893e76872a None admin] PATH_INFO: `/v3/projects/a6074dbdaa494f65bb96b84b2ad9e4fd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:47.362313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414c606-ef3f-4908-8f23-cb893e76872a None admin] RBAC: Authorizing `identity:get_project(project_id=a6074dbdaa494f65bb96b84b2ad9e4fd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:47.363886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414c606-ef3f-4908-8f23-cb893e76872a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:47.365676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 573/1145] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:13:47 2022] GET /identity/v3/projects/a6074dbdaa494f65bb96b84b2ad9e4fd => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:47.372621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf8562-b280-4e32-b2ee-20985618a895 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:47.491164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf8562-b280-4e32-b2ee-20985618a895 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:47.551921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f1faf9-b241-4302-b799-0ab61e4a5e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:47.552995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1faf9-b241-4302-b799-0ab61e4a5e99 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:47.553502 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1faf9-b241-4302-b799-0ab61e4a5e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:47.554039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f1faf9-b241-4302-b799-0ab61e4a5e99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:47.555268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1faf9-b241-4302-b799-0ab61e4a5e99 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=a6074dbdaa494f65bb96b84b2ad9e4fd, user.id=a321656c417d4a398b229009a2f1e3a6) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:47.555837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1faf9-b241-4302-b799-0ab61e4a5e99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:47.558569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f1faf9-b241-4302-b799-0ab61e4a5e99 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:47.568431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 573/1146] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:13:47 2022] GET /identity/v3/role_assignments?user.id=a321656c417d4a398b229009a2f1e3a6&scope.project.id=a6074dbdaa494f65bb96b84b2ad9e4fd&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:49.580758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d705b-6add-429f-bc1e-83fd3fe1dee7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:49.581363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d705b-6add-429f-bc1e-83fd3fe1dee7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:49.581363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d705b-6add-429f-bc1e-83fd3fe1dee7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:49.583245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 574/1147] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:49.589370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95420b4-f309-4b2e-a104-3eaf791b3484 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:49.589916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95420b4-f309-4b2e-a104-3eaf791b3484 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:49.590052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95420b4-f309-4b2e-a104-3eaf791b3484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:49.597197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:49.612955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a95420b4-f309-4b2e-a104-3eaf791b3484 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:49.614298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a95420b4-f309-4b2e-a104-3eaf791b3484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:49.688613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 574/1148] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:49.712154 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d705b-6add-429f-bc1e-83fd3fe1dee7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:49.823182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d705b-6add-429f-bc1e-83fd3fe1dee7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:49.863883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40c7c7-2cd8-4803-97a1-ffa88084a2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:49.864589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40c7c7-2cd8-4803-97a1-ffa88084a2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:49.864910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40c7c7-2cd8-4803-97a1-ffa88084a2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:49.865232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b40c7c7-2cd8-4803-97a1-ffa88084a2a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:49.877558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b40c7c7-2cd8-4803-97a1-ffa88084a2a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5b40c7c7-2cd8-4803-97a1-ffa88084a2a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:13:49.882796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:49.884599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:49.886208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 575/1149] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:13:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:49.892009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95420b4-f309-4b2e-a104-3eaf791b3484 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:50.004811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95420b4-f309-4b2e-a104-3eaf791b3484 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:50.038335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:50.046881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7885731-9877-42dd-8f44-c1d20abd2356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:50.047981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7885731-9877-42dd-8f44-c1d20abd2356 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:50.048470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7885731-9877-42dd-8f44-c1d20abd2356 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:50.048954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7885731-9877-42dd-8f44-c1d20abd2356 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:50.050337 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7885731-9877-42dd-8f44-c1d20abd2356 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:50.050880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7885731-9877-42dd-8f44-c1d20abd2356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:50.053554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7885731-9877-42dd-8f44-c1d20abd2356 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:50.070249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 575/1150] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:13:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:50.076455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40c7c7-2cd8-4803-97a1-ffa88084a2a7 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:50.164143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:50.230772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b40c7c7-2cd8-4803-97a1-ffa88084a2a7 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:50.268495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c44d2-ffaa-4689-b435-48eb2c4b8d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:50.269110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c44d2-ffaa-4689-b435-48eb2c4b8d2a None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:50.269435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c44d2-ffaa-4689-b435-48eb2c4b8d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:50.269809 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189c44d2-ffaa-4689-b435-48eb2c4b8d2a None admin] PATH_INFO: `/v3/users/a321656c417d4a398b229009a2f1e3a6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:50.271179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c44d2-ffaa-4689-b435-48eb2c4b8d2a None admin] RBAC: Authorizing `identity:get_user(user_id=a321656c417d4a398b229009a2f1e3a6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:50.272678 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189c44d2-ffaa-4689-b435-48eb2c4b8d2a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:50.274391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 576/1151] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:50 2022] GET /identity/v3/users/a321656c417d4a398b229009a2f1e3a6 => generated 286 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:50.280656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7885731-9877-42dd-8f44-c1d20abd2356 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:50.388928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7885731-9877-42dd-8f44-c1d20abd2356 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:50.429582 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167d1fe4-e1e3-451d-953e-1b5dfca17ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:50.430557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d1fe4-e1e3-451d-953e-1b5dfca17ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:50.430922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d1fe4-e1e3-451d-953e-1b5dfca17ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:50.431297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167d1fe4-e1e3-451d-953e-1b5dfca17ef8 None admin] PATH_INFO: `/v3/projects/a6074dbdaa494f65bb96b84b2ad9e4fd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:50.432788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d1fe4-e1e3-451d-953e-1b5dfca17ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=a6074dbdaa494f65bb96b84b2ad9e4fd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:50.434354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167d1fe4-e1e3-451d-953e-1b5dfca17ef8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:50.436173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 576/1152] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:13:50 2022] GET /identity/v3/projects/a6074dbdaa494f65bb96b84b2ad9e4fd => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:50.442698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c44d2-ffaa-4689-b435-48eb2c4b8d2a None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:50.563856 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189c44d2-ffaa-4689-b435-48eb2c4b8d2a None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:50.581695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:50.601693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a671c15c-d25e-41b9-961f-a3c4b068c23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:50.602416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671c15c-d25e-41b9-961f-a3c4b068c23a None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:50.602698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671c15c-d25e-41b9-961f-a3c4b068c23a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:50.602970 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a671c15c-d25e-41b9-961f-a3c4b068c23a None admin] PATH_INFO: `/v3/projects/a6074dbdaa494f65bb96b84b2ad9e4fd/users/a321656c417d4a398b229009a2f1e3a6/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:50.604973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671c15c-d25e-41b9-961f-a3c4b068c23a None admin] RBAC: Authorizing `identity:create_grant(project_id=a6074dbdaa494f65bb96b84b2ad9e4fd, user_id=a321656c417d4a398b229009a2f1e3a6, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:50.606458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a671c15c-d25e-41b9-961f-a3c4b068c23a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:50.622241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 577/1153] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:13:50 2022] PUT /identity/v3/projects/a6074dbdaa494f65bb96b84b2ad9e4fd/users/a321656c417d4a398b229009a2f1e3a6/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:13:52.655282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a1451-13aa-451a-8ac7-051711c412fd None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:52.655916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a1451-13aa-451a-8ac7-051711c412fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:52.655916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a1451-13aa-451a-8ac7-051711c412fd None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:52.657171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 577/1154] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:13:52.663315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f08a7-8611-4af0-a93a-897ab8d83b0b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:52.663868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f08a7-8611-4af0-a93a-897ab8d83b0b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:52.663868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f08a7-8611-4af0-a93a-897ab8d83b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:52.683951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-715f08a7-8611-4af0-a93a-897ab8d83b0b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:52.685171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-715f08a7-8611-4af0-a93a-897ab8d83b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:52.760822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 578/1155] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:52.784578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a1451-13aa-451a-8ac7-051711c412fd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:52.849056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:52.895921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a1451-13aa-451a-8ac7-051711c412fd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:52.935983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fa9c1-111c-4df1-baa5-85028dda3867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:52.936645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fa9c1-111c-4df1-baa5-85028dda3867 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:52.936928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fa9c1-111c-4df1-baa5-85028dda3867 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:52.937199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fa9c1-111c-4df1-baa5-85028dda3867 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:52.950646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fa9c1-111c-4df1-baa5-85028dda3867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-b46fa9c1-111c-4df1-baa5-85028dda3867 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:13:52.955916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:52.957141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:13:52.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 578/1156] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:13:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:13:52.964269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f08a7-8611-4af0-a93a-897ab8d83b0b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:53.075736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f08a7-8611-4af0-a93a-897ab8d83b0b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:53.084103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:53.114549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c84f5b-7c89-41a9-8cce-7d5cb07a95d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:53.115179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84f5b-7c89-41a9-8cce-7d5cb07a95d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:53.115458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84f5b-7c89-41a9-8cce-7d5cb07a95d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:53.115729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c84f5b-7c89-41a9-8cce-7d5cb07a95d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:53.116454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c84f5b-7c89-41a9-8cce-7d5cb07a95d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:53.116761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c84f5b-7c89-41a9-8cce-7d5cb07a95d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:53.118160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c84f5b-7c89-41a9-8cce-7d5cb07a95d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:53.127018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 579/1157] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:53.134539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fa9c1-111c-4df1-baa5-85028dda3867 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:53.246875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fa9c1-111c-4df1-baa5-85028dda3867 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:53.270553 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:53.283820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edb7ab-8730-4eeb-8bfa-36f4771a3f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:53.284478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edb7ab-8730-4eeb-8bfa-36f4771a3f0c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:53.284757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edb7ab-8730-4eeb-8bfa-36f4771a3f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:53.284999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68edb7ab-8730-4eeb-8bfa-36f4771a3f0c None admin] PATH_INFO: `/v3/users/a321656c417d4a398b229009a2f1e3a6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:53.286390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edb7ab-8730-4eeb-8bfa-36f4771a3f0c None admin] RBAC: Authorizing `identity:get_user(user_id=a321656c417d4a398b229009a2f1e3a6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:53.287833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68edb7ab-8730-4eeb-8bfa-36f4771a3f0c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:53.289458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 579/1158] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:13:53 2022] GET /identity/v3/users/a321656c417d4a398b229009a2f1e3a6 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:13:53.295501 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c84f5b-7c89-41a9-8cce-7d5cb07a95d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:53.406119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c84f5b-7c89-41a9-8cce-7d5cb07a95d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:53.444909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6860087-f6d8-4bcc-99b9-47f2ef57eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:53.445609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6860087-f6d8-4bcc-99b9-47f2ef57eca8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:53.445983 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6860087-f6d8-4bcc-99b9-47f2ef57eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:53.446307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6860087-f6d8-4bcc-99b9-47f2ef57eca8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:53.483252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:53.517945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6860087-f6d8-4bcc-99b9-47f2ef57eca8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:53.519395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6860087-f6d8-4bcc-99b9-47f2ef57eca8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:53.627737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 580/1159] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:13:53.634092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edb7ab-8730-4eeb-8bfa-36f4771a3f0c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:53.748327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68edb7ab-8730-4eeb-8bfa-36f4771a3f0c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:53.785290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c91584-2178-474c-996e-fa0bfc65b21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:53.786050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c91584-2178-474c-996e-fa0bfc65b21d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:53.786360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c91584-2178-474c-996e-fa0bfc65b21d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:53.786676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c91584-2178-474c-996e-fa0bfc65b21d None admin] PATH_INFO: `/v3/projects/a6074dbdaa494f65bb96b84b2ad9e4fd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:53.787922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c91584-2178-474c-996e-fa0bfc65b21d None admin] RBAC: Authorizing `identity:get_project(project_id=a6074dbdaa494f65bb96b84b2ad9e4fd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:53.789413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c91584-2178-474c-996e-fa0bfc65b21d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:53.791117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 580/1160] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:13:53 2022] GET /identity/v3/projects/a6074dbdaa494f65bb96b84b2ad9e4fd => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:53.800810 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6860087-f6d8-4bcc-99b9-47f2ef57eca8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:53.875380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:53.916750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6860087-f6d8-4bcc-99b9-47f2ef57eca8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:53.957589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a41924-039e-4620-bbe8-cb81b20b9de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:53.958291 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a41924-039e-4620-bbe8-cb81b20b9de8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:53.958572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a41924-039e-4620-bbe8-cb81b20b9de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:53.958936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a41924-039e-4620-bbe8-cb81b20b9de8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:53.959774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a41924-039e-4620-bbe8-cb81b20b9de8 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=a6074dbdaa494f65bb96b84b2ad9e4fd, user.id=a321656c417d4a398b229009a2f1e3a6) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:53.960082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a41924-039e-4620-bbe8-cb81b20b9de8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:53.961613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a41924-039e-4620-bbe8-cb81b20b9de8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:53.968896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 581/1161] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:13:53 2022] GET /identity/v3/role_assignments?user.id=a321656c417d4a398b229009a2f1e3a6&scope.project.id=a6074dbdaa494f65bb96b84b2ad9e4fd&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:56.040597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df9f10-2117-4418-a6b8-ab0776ea60b4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:56.041277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df9f10-2117-4418-a6b8-ab0776ea60b4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:56.041277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df9f10-2117-4418-a6b8-ab0776ea60b4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:56.042721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 581/1162] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:56.048574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcde03d-b5d1-4d72-8d3e-097b91aa0a31 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:56.049111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcde03d-b5d1-4d72-8d3e-097b91aa0a31 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:56.049111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcde03d-b5d1-4d72-8d3e-097b91aa0a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:56.072541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8dcde03d-b5d1-4d72-8d3e-097b91aa0a31 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:56.073805 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8dcde03d-b5d1-4d72-8d3e-097b91aa0a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:56.163635 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 582/1163] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:56.186085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df9f10-2117-4418-a6b8-ab0776ea60b4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:56.248976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:56.297066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df9f10-2117-4418-a6b8-ab0776ea60b4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:56.333882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84117bc-d078-49ec-853d-25bb510ee7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:56.334573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84117bc-d078-49ec-853d-25bb510ee7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:56.334894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84117bc-d078-49ec-853d-25bb510ee7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:56.335209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84117bc-d078-49ec-853d-25bb510ee7e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:56.409412 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84117bc-d078-49ec-853d-25bb510ee7e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:56.410970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84117bc-d078-49ec-853d-25bb510ee7e3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:56.472884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:56.521347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 582/1164] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:13:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:13:56.527797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcde03d-b5d1-4d72-8d3e-097b91aa0a31 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:56.595328 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:56.640860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcde03d-b5d1-4d72-8d3e-097b91aa0a31 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:56.677898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379c46f8-200f-4515-9e1d-9ed07805b325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:56.678561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c46f8-200f-4515-9e1d-9ed07805b325 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:56.678843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c46f8-200f-4515-9e1d-9ed07805b325 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:56.679118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c46f8-200f-4515-9e1d-9ed07805b325 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:56.686391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c46f8-200f-4515-9e1d-9ed07805b325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:56.687850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379c46f8-200f-4515-9e1d-9ed07805b325 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-379c46f8-200f-4515-9e1d-9ed07805b325 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:13:56.700119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:13:56.702161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:13:56.703989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 583/1165] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jul 26 07:13:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:13:56.709941 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84117bc-d078-49ec-853d-25bb510ee7e3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:56.820387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84117bc-d078-49ec-853d-25bb510ee7e3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:56.861616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7a7a5-1ccb-484c-a876-e8ed193a67ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:56.862215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7a7a5-1ccb-484c-a876-e8ed193a67ac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:56.862499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7a7a5-1ccb-484c-a876-e8ed193a67ac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:56.862768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7a7a5-1ccb-484c-a876-e8ed193a67ac None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:56.863526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7a7a5-1ccb-484c-a876-e8ed193a67ac None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:13:56.863831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7a7a5-1ccb-484c-a876-e8ed193a67ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:56.865281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7a7a5-1ccb-484c-a876-e8ed193a67ac None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:56.874257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 583/1166] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Jul 26 07:13:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:13:59.008323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ef58a-c690-43a3-b2dd-80daf09ae2c1 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:59.008930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ef58a-c690-43a3-b2dd-80daf09ae2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:59.008930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ef58a-c690-43a3-b2dd-80daf09ae2c1 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:59.010516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 584/1167] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:13:59.018399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77fa2a-244a-48bd-8480-fe1ace5aeccb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:59.019279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77fa2a-244a-48bd-8480-fe1ace5aeccb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:59.019279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d77fa2a-244a-48bd-8480-fe1ace5aeccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:59.038732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4d77fa2a-244a-48bd-8480-fe1ace5aeccb None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:13:59.039931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4d77fa2a-244a-48bd-8480-fe1ace5aeccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:13:59.058256 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:59.113828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 584/1168] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:13:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:13:59.137135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ef58a-c690-43a3-b2dd-80daf09ae2c1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:13:59.246949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ef58a-c690-43a3-b2dd-80daf09ae2c1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:13:59.271435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:13:59.290636 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5a0ad-06e9-4c98-bd80-d1121390a743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:13:59.291675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5a0ad-06e9-4c98-bd80-d1121390a743 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:13:59.292136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5a0ad-06e9-4c98-bd80-d1121390a743 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:13:59.292623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5a0ad-06e9-4c98-bd80-d1121390a743 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:13:59.293993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5a0ad-06e9-4c98-bd80-d1121390a743 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:13:59.296437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5a0ad-06e9-4c98-bd80-d1121390a743 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:13:59.318693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 585/1169] 10.222.0.39 () {60 vars in 1138 bytes} [Tue Jul 26 07:13:59 2022] POST /identity/v3/domains => generated 267 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:14:01.367439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5ce9b-c26e-47f8-afe2-2a488e9043ca None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:01.368006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5ce9b-c26e-47f8-afe2-2a488e9043ca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:01.368006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c5ce9b-c26e-47f8-afe2-2a488e9043ca None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:01.369486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 585/1170] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:14:01.375657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb9f90-b9be-4ebf-921d-fb9f984cf397 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:01.376218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb9f90-b9be-4ebf-921d-fb9f984cf397 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:01.376327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb9f90-b9be-4ebf-921d-fb9f984cf397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:01.396458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c0eb9f90-b9be-4ebf-921d-fb9f984cf397 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:14:01.397757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0eb9f90-b9be-4ebf-921d-fb9f984cf397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:14:01.476566 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 586/1171] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:14:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:14:01.500614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5ce9b-c26e-47f8-afe2-2a488e9043ca None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:01.652064 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c5ce9b-c26e-47f8-afe2-2a488e9043ca None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:01.690575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2229a6-dbd0-4b9f-bb34-dc27e33d099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:01.691246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2229a6-dbd0-4b9f-bb34-dc27e33d099b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:01.691574 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2229a6-dbd0-4b9f-bb34-dc27e33d099b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:01.691850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2229a6-dbd0-4b9f-bb34-dc27e33d099b None admin] PATH_INFO: `/v3/domains/d1d01d6838604224ae6be67523e2d450` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:01.703948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2229a6-dbd0-4b9f-bb34-dc27e33d099b None admin] RBAC: Authorizing `identity:get_domain(domain_id=d1d01d6838604224ae6be67523e2d450)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:01.705425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2229a6-dbd0-4b9f-bb34-dc27e33d099b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:01.707327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 586/1172] 10.222.0.39 () {56 vars in 1183 bytes} [Tue Jul 26 07:14:01 2022] GET /identity/v3/domains/d1d01d6838604224ae6be67523e2d450 => generated 267 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:01.713728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb9f90-b9be-4ebf-921d-fb9f984cf397 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:01.822512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb9f90-b9be-4ebf-921d-fb9f984cf397 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:01.837639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:01.863519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708bf44-13fd-4d80-a3b4-9e221f60b7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:01.864165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708bf44-13fd-4d80-a3b4-9e221f60b7cf None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:01.864446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708bf44-13fd-4d80-a3b4-9e221f60b7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:01.864719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708bf44-13fd-4d80-a3b4-9e221f60b7cf None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:01.865433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708bf44-13fd-4d80-a3b4-9e221f60b7cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:01.866933 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708bf44-13fd-4d80-a3b4-9e221f60b7cf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:01.898222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 587/1173] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:14:01 2022] POST /identity/v3/projects => generated 361 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:14:04.383448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5414f27-679b-4398-8709-49e4d7c0b194 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:04.384055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5414f27-679b-4398-8709-49e4d7c0b194 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:04.384055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5414f27-679b-4398-8709-49e4d7c0b194 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:04.385388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 587/1174] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:14:04.391523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3b5b7-5873-4b0f-a8f7-c8a63ff87929 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:04.392076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3b5b7-5873-4b0f-a8f7-c8a63ff87929 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:04.392218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3b5b7-5873-4b0f-a8f7-c8a63ff87929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:04.411957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5d3b5b7-5873-4b0f-a8f7-c8a63ff87929 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:14:04.413185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5d3b5b7-5873-4b0f-a8f7-c8a63ff87929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:14:04.489591 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 588/1175] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:14:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:14:04.514025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5414f27-679b-4398-8709-49e4d7c0b194 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:04.532385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:04.636850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5414f27-679b-4398-8709-49e4d7c0b194 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:04.675534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b2646-bd53-461e-b76a-1d4d1ce900ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:04.676326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2646-bd53-461e-b76a-1d4d1ce900ac None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:04.676608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2646-bd53-461e-b76a-1d4d1ce900ac None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:04.676888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b2646-bd53-461e-b76a-1d4d1ce900ac None admin] PATH_INFO: `/v3/domains/d1d01d6838604224ae6be67523e2d450` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:04.678233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b2646-bd53-461e-b76a-1d4d1ce900ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=d1d01d6838604224ae6be67523e2d450)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:04.679706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b2646-bd53-461e-b76a-1d4d1ce900ac None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:04.681368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 588/1176] 10.222.0.39 () {56 vars in 1183 bytes} [Tue Jul 26 07:14:04 2022] GET /identity/v3/domains/d1d01d6838604224ae6be67523e2d450 => generated 267 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:04.687629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3b5b7-5873-4b0f-a8f7-c8a63ff87929 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:04.794759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:04.802972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3b5b7-5873-4b0f-a8f7-c8a63ff87929 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:04.843044 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77a4ac5-c382-428b-826e-81ec32ea7e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:04.843751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4ac5-c382-428b-826e-81ec32ea7e69 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:04.844081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4ac5-c382-428b-826e-81ec32ea7e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:04.844397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77a4ac5-c382-428b-826e-81ec32ea7e69 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:04.845159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a4ac5-c382-428b-826e-81ec32ea7e69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:04.846685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77a4ac5-c382-428b-826e-81ec32ea7e69 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:04.871946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 589/1177] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:14:04 2022] POST /identity/v3/users => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:14:06.978308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796acfc0-9d68-4333-b40e-4c9251019336 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:06.978922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796acfc0-9d68-4333-b40e-4c9251019336 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:06.979224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796acfc0-9d68-4333-b40e-4c9251019336 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:06.981525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 589/1178] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:14:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:14:06.990344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126c9a1-61e2-4bb9-bf9f-ec6fdafd0784 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:06.991111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126c9a1-61e2-4bb9-bf9f-ec6fdafd0784 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:06.991281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1126c9a1-61e2-4bb9-bf9f-ec6fdafd0784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:07.010792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1126c9a1-61e2-4bb9-bf9f-ec6fdafd0784 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:14:07.012047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1126c9a1-61e2-4bb9-bf9f-ec6fdafd0784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:14:07.087383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 590/1179] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:14:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:14:07.111226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796acfc0-9d68-4333-b40e-4c9251019336 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:07.201895 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:07.226265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796acfc0-9d68-4333-b40e-4c9251019336 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:07.264450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9310e3-4e95-4666-a411-b4ec8d1f7810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:07.265113 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9310e3-4e95-4666-a411-b4ec8d1f7810 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:07.265397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9310e3-4e95-4666-a411-b4ec8d1f7810 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:07.265671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9310e3-4e95-4666-a411-b4ec8d1f7810 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:07.277625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9310e3-4e95-4666-a411-b4ec8d1f7810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-bf9310e3-4e95-4666-a411-b4ec8d1f7810 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:14:07.282615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:14:07.283645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:14:07.284505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 590/1180] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:14:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:14:07.290926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126c9a1-61e2-4bb9-bf9f-ec6fdafd0784 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:07.378097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:07.401879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1126c9a1-61e2-4bb9-bf9f-ec6fdafd0784 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:07.440293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487011e3-97a3-4722-980a-6c17635189b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:07.441042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487011e3-97a3-4722-980a-6c17635189b6 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:07.441361 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487011e3-97a3-4722-980a-6c17635189b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:07.441842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487011e3-97a3-4722-980a-6c17635189b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:07.442633 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487011e3-97a3-4722-980a-6c17635189b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:14:07.442982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487011e3-97a3-4722-980a-6c17635189b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:07.444428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487011e3-97a3-4722-980a-6c17635189b6 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:07.453884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 591/1181] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:14:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:07.460206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9310e3-4e95-4666-a411-b4ec8d1f7810 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:07.569962 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9310e3-4e95-4666-a411-b4ec8d1f7810 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:07.605760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dff83-5996-4eb7-bbea-517976fb5960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:07.606454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dff83-5996-4eb7-bbea-517976fb5960 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:07.606735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dff83-5996-4eb7-bbea-517976fb5960 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:07.607008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027dff83-5996-4eb7-bbea-517976fb5960 None admin] PATH_INFO: `/v3/users/04612b5b058240eea9f2f8c8ef8c5130` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:07.619448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:07.623612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dff83-5996-4eb7-bbea-517976fb5960 None admin] RBAC: Authorizing `identity:get_user(user_id=04612b5b058240eea9f2f8c8ef8c5130)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:07.625432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027dff83-5996-4eb7-bbea-517976fb5960 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:07.627286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 591/1182] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:14:07 2022] GET /identity/v3/users/04612b5b058240eea9f2f8c8ef8c5130 => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:07.633342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487011e3-97a3-4722-980a-6c17635189b6 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:07.741424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487011e3-97a3-4722-980a-6c17635189b6 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:07.779860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06e34e-1515-48ee-b7f0-a3367ae938e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:07.780503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06e34e-1515-48ee-b7f0-a3367ae938e3 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:07.780779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06e34e-1515-48ee-b7f0-a3367ae938e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:07.781053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06e34e-1515-48ee-b7f0-a3367ae938e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:07.796260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:07.872577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06e34e-1515-48ee-b7f0-a3367ae938e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:07.874114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06e34e-1515-48ee-b7f0-a3367ae938e3 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:08.007924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 592/1183] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:14:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:14:08.014232 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dff83-5996-4eb7-bbea-517976fb5960 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:08.124701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027dff83-5996-4eb7-bbea-517976fb5960 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:08.180418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86faedfe-e384-43ab-ae27-e96c76e7185b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:08.181088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faedfe-e384-43ab-ae27-e96c76e7185b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:08.181370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faedfe-e384-43ab-ae27-e96c76e7185b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:08.181644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86faedfe-e384-43ab-ae27-e96c76e7185b None admin] PATH_INFO: `/v3/projects/57529f25ceba4e3dab661cbc0bf024fb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:08.182876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86faedfe-e384-43ab-ae27-e96c76e7185b None admin] RBAC: Authorizing `identity:get_project(project_id=57529f25ceba4e3dab661cbc0bf024fb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:08.184347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86faedfe-e384-43ab-ae27-e96c76e7185b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:08.186116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 592/1184] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:14:08 2022] GET /identity/v3/projects/57529f25ceba4e3dab661cbc0bf024fb => generated 361 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:08.192778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06e34e-1515-48ee-b7f0-a3367ae938e3 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:08.240118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:08.304792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06e34e-1515-48ee-b7f0-a3367ae938e3 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:08.343004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc0b795-1e93-4230-9948-e547f1e471cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:08.343642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0b795-1e93-4230-9948-e547f1e471cd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:08.343927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0b795-1e93-4230-9948-e547f1e471cd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:08.344204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc0b795-1e93-4230-9948-e547f1e471cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:08.345012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0b795-1e93-4230-9948-e547f1e471cd None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=57529f25ceba4e3dab661cbc0bf024fb, user.id=04612b5b058240eea9f2f8c8ef8c5130) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:14:08.345325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0b795-1e93-4230-9948-e547f1e471cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:08.346814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc0b795-1e93-4230-9948-e547f1e471cd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:08.353125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 593/1185] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:14:08 2022] GET /identity/v3/role_assignments?user.id=04612b5b058240eea9f2f8c8ef8c5130&scope.project.id=57529f25ceba4e3dab661cbc0bf024fb&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:10.390132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbed763-e268-4cc2-ba58-d1326f9c7f1d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:10.390982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbed763-e268-4cc2-ba58-d1326f9c7f1d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:10.390982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbed763-e268-4cc2-ba58-d1326f9c7f1d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:10.392517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 593/1186] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:14:10.398529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba1781-0aef-46c2-b228-59b12e054949 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:10.399087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba1781-0aef-46c2-b228-59b12e054949 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:10.399087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ba1781-0aef-46c2-b228-59b12e054949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:10.419749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-57ba1781-0aef-46c2-b228-59b12e054949 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:14:10.421091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-57ba1781-0aef-46c2-b228-59b12e054949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:14:10.495853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 594/1187] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:14:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:14:10.518939 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbed763-e268-4cc2-ba58-d1326f9c7f1d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:10.589818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:10.628817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbed763-e268-4cc2-ba58-d1326f9c7f1d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:10.665437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c1d7c-2fd0-445e-9d9a-fc4060b279ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:10.666016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c1d7c-2fd0-445e-9d9a-fc4060b279ec None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:10.666296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c1d7c-2fd0-445e-9d9a-fc4060b279ec None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:10.666565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c1d7c-2fd0-445e-9d9a-fc4060b279ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:10.678084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c1d7c-2fd0-445e-9d9a-fc4060b279ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8d3c1d7c-2fd0-445e-9d9a-fc4060b279ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:14:10.683065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:14:10.689713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:14:10.690830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 594/1188] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:14:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:14:10.691730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba1781-0aef-46c2-b228-59b12e054949 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:10.730405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:10.823700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ba1781-0aef-46c2-b228-59b12e054949 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:10.864634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14238ee4-0ccb-42b5-860b-5c99082b38b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:10.865262 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14238ee4-0ccb-42b5-860b-5c99082b38b8 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:10.865544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14238ee4-0ccb-42b5-860b-5c99082b38b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:10.865821 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14238ee4-0ccb-42b5-860b-5c99082b38b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:10.866585 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14238ee4-0ccb-42b5-860b-5c99082b38b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:14:10.866858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14238ee4-0ccb-42b5-860b-5c99082b38b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:10.868261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14238ee4-0ccb-42b5-860b-5c99082b38b8 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:10.877935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 595/1189] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:14:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:14:10.884929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c1d7c-2fd0-445e-9d9a-fc4060b279ec None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:11.018468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c1d7c-2fd0-445e-9d9a-fc4060b279ec None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:11.048039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:11.058193 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c6189-5d0a-44b3-ba3f-0fd113a5c6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:11.058858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c6189-5d0a-44b3-ba3f-0fd113a5c6ab None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:11.059137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c6189-5d0a-44b3-ba3f-0fd113a5c6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:11.059410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c6189-5d0a-44b3-ba3f-0fd113a5c6ab None admin] PATH_INFO: `/v3/users/04612b5b058240eea9f2f8c8ef8c5130` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:11.060600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c6189-5d0a-44b3-ba3f-0fd113a5c6ab None admin] RBAC: Authorizing `identity:get_user(user_id=04612b5b058240eea9f2f8c8ef8c5130)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:11.062091 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c6189-5d0a-44b3-ba3f-0fd113a5c6ab None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:11.063747 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 595/1190] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:14:10 2022] GET /identity/v3/users/04612b5b058240eea9f2f8c8ef8c5130 => generated 311 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:11.072526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14238ee4-0ccb-42b5-860b-5c99082b38b8 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:11.185699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:11.193228 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14238ee4-0ccb-42b5-860b-5c99082b38b8 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:11.257068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d9a1c0-ddfc-4679-81d7-fdd98ea6ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:11.258234 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9a1c0-ddfc-4679-81d7-fdd98ea6ab0e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:11.258686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9a1c0-ddfc-4679-81d7-fdd98ea6ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:11.259109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d9a1c0-ddfc-4679-81d7-fdd98ea6ab0e None admin] PATH_INFO: `/v3/projects/57529f25ceba4e3dab661cbc0bf024fb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:11.260998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9a1c0-ddfc-4679-81d7-fdd98ea6ab0e None admin] RBAC: Authorizing `identity:get_project(project_id=57529f25ceba4e3dab661cbc0bf024fb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:11.263444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d9a1c0-ddfc-4679-81d7-fdd98ea6ab0e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:11.266454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 596/1191] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:14:11 2022] GET /identity/v3/projects/57529f25ceba4e3dab661cbc0bf024fb => generated 361 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:11.273397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c6189-5d0a-44b3-ba3f-0fd113a5c6ab None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:11.401201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c6189-5d0a-44b3-ba3f-0fd113a5c6ab None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:11.443033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1262b5e-1b83-4f3d-b64b-34523b265156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:11.443740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1262b5e-1b83-4f3d-b64b-34523b265156 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:11.444017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1262b5e-1b83-4f3d-b64b-34523b265156 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:11.444285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1262b5e-1b83-4f3d-b64b-34523b265156 None admin] PATH_INFO: `/v3/projects/57529f25ceba4e3dab661cbc0bf024fb/users/04612b5b058240eea9f2f8c8ef8c5130/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:11.446413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1262b5e-1b83-4f3d-b64b-34523b265156 None admin] RBAC: Authorizing `identity:create_grant(project_id=57529f25ceba4e3dab661cbc0bf024fb, user_id=04612b5b058240eea9f2f8c8ef8c5130, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:11.447895 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1262b5e-1b83-4f3d-b64b-34523b265156 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:11.463697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 596/1192] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:14:11 2022] PUT /identity/v3/projects/57529f25ceba4e3dab661cbc0bf024fb/users/04612b5b058240eea9f2f8c8ef8c5130/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:14:13.512859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362f42-5f49-42f8-98de-15f1f3fdde99 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:13.513509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362f42-5f49-42f8-98de-15f1f3fdde99 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:13.513509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4362f42-5f49-42f8-98de-15f1f3fdde99 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:13.514823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 597/1193] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:14:13.520308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c7935-6c20-4184-b542-3bc506efffe1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:13.520764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c7935-6c20-4184-b542-3bc506efffe1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:13.520868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c7935-6c20-4184-b542-3bc506efffe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:13.539978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-954c7935-6c20-4184-b542-3bc506efffe1 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:14:13.541168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-954c7935-6c20-4184-b542-3bc506efffe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:14:13.592639 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:13.613418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 597/1194] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:14:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:14:13.636691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362f42-5f49-42f8-98de-15f1f3fdde99 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:13.756735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4362f42-5f49-42f8-98de-15f1f3fdde99 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:13.790721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd2036-3779-4c13-bec9-7d15b7df2e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:13.791367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd2036-3779-4c13-bec9-7d15b7df2e5d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:13.791646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd2036-3779-4c13-bec9-7d15b7df2e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:13.791918 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd2036-3779-4c13-bec9-7d15b7df2e5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:13.803979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd2036-3779-4c13-bec9-7d15b7df2e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f8bd2036-3779-4c13-bec9-7d15b7df2e5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:14:13.808896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:14:13.810046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:14:13.811108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 598/1195] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:14:13.816834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c7935-6c20-4184-b542-3bc506efffe1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:13.942448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c7935-6c20-4184-b542-3bc506efffe1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:13.992077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe74e1f-12c3-460f-af49-ae1145ad281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:13.992710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe74e1f-12c3-460f-af49-ae1145ad281b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:13.992996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe74e1f-12c3-460f-af49-ae1145ad281b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:13.993267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe74e1f-12c3-460f-af49-ae1145ad281b None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:13.994022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe74e1f-12c3-460f-af49-ae1145ad281b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:14:13.994322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe74e1f-12c3-460f-af49-ae1145ad281b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:13.995698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe74e1f-12c3-460f-af49-ae1145ad281b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:14.004856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 598/1196] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:14:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:14.013180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd2036-3779-4c13-bec9-7d15b7df2e5d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:14.048609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:14.153394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd2036-3779-4c13-bec9-7d15b7df2e5d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:14.191215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f73e2-eba5-4296-93d3-46efb72af661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:14.191947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f73e2-eba5-4296-93d3-46efb72af661 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:14.192230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f73e2-eba5-4296-93d3-46efb72af661 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:14.192504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f73e2-eba5-4296-93d3-46efb72af661 None admin] PATH_INFO: `/v3/users/04612b5b058240eea9f2f8c8ef8c5130` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:14.193692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f73e2-eba5-4296-93d3-46efb72af661 None admin] RBAC: Authorizing `identity:get_user(user_id=04612b5b058240eea9f2f8c8ef8c5130)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:14.195158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f73e2-eba5-4296-93d3-46efb72af661 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:14.196825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 599/1197] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Jul 26 07:14:14 2022] GET /identity/v3/users/04612b5b058240eea9f2f8c8ef8c5130 => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:14.203120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe74e1f-12c3-460f-af49-ae1145ad281b None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:14.254392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:14.311242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe74e1f-12c3-460f-af49-ae1145ad281b None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:14.347664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b928-a6f8-4946-b9ee-aeaa306aab42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:14.348395 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b928-a6f8-4946-b9ee-aeaa306aab42 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:14.348722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b928-a6f8-4946-b9ee-aeaa306aab42 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:14.349047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b928-a6f8-4946-b9ee-aeaa306aab42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:14.421678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686b928-a6f8-4946-b9ee-aeaa306aab42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:14.423250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686b928-a6f8-4946-b9ee-aeaa306aab42 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:14.471920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:14.527522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 599/1198] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:14:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 07:14:14.533861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f73e2-eba5-4296-93d3-46efb72af661 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:14.633350 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:14.640994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f73e2-eba5-4296-93d3-46efb72af661 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:14.675731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dd4c43-467d-47aa-8355-5c031732b5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:14.676453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd4c43-467d-47aa-8355-5c031732b5e0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:14.676730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd4c43-467d-47aa-8355-5c031732b5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:14.677005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dd4c43-467d-47aa-8355-5c031732b5e0 None admin] PATH_INFO: `/v3/projects/57529f25ceba4e3dab661cbc0bf024fb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:14.678231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd4c43-467d-47aa-8355-5c031732b5e0 None admin] RBAC: Authorizing `identity:get_project(project_id=57529f25ceba4e3dab661cbc0bf024fb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:14.679669 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dd4c43-467d-47aa-8355-5c031732b5e0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:14.681365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 600/1199] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Jul 26 07:14:14 2022] GET /identity/v3/projects/57529f25ceba4e3dab661cbc0bf024fb => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:14:14.687796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b928-a6f8-4946-b9ee-aeaa306aab42 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:14:14.794327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b928-a6f8-4946-b9ee-aeaa306aab42 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:14:14.831534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817212c-fd5c-4247-bfdd-562885061c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:14:14.832170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817212c-fd5c-4247-bfdd-562885061c27 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:14:14.832485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817212c-fd5c-4247-bfdd-562885061c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:14:14.832756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817212c-fd5c-4247-bfdd-562885061c27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:14:14.833547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817212c-fd5c-4247-bfdd-562885061c27 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=57529f25ceba4e3dab661cbc0bf024fb, user.id=04612b5b058240eea9f2f8c8ef8c5130) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:14:14.833892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817212c-fd5c-4247-bfdd-562885061c27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:14:14.835335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3817212c-fd5c-4247-bfdd-562885061c27 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:14:14.841650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 600/1200] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:14:14 2022] GET /identity/v3/role_assignments?user.id=04612b5b058240eea9f2f8c8ef8c5130&scope.project.id=57529f25ceba4e3dab661cbc0bf024fb&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:14:24.671695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=19 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:14:24.839554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:13.677389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285e685-1f18-47e2-b620-b284bec1d31f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:13.678482 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285e685-1f18-47e2-b620-b284bec1d31f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:13.678482 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285e685-1f18-47e2-b620-b284bec1d31f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:13.681105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 601/1201] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:15:13.689224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7bbc2-127b-4dd7-9ce2-60805c77d08d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:13.690887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7bbc2-127b-4dd7-9ce2-60805c77d08d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:13.691731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7bbc2-127b-4dd7-9ce2-60805c77d08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:13.731598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-21f7bbc2-127b-4dd7-9ce2-60805c77d08d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:13.733715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-21f7bbc2-127b-4dd7-9ce2-60805c77d08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:13.860599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 601/1202] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:15:13.908681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285e685-1f18-47e2-b620-b284bec1d31f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:14.085178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285e685-1f18-47e2-b620-b284bec1d31f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:14.150688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4e954-6f6e-4a8f-af64-73fca8def346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:14.151794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4e954-6f6e-4a8f-af64-73fca8def346 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:14.152286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4e954-6f6e-4a8f-af64-73fca8def346 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:14.152774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff4e954-6f6e-4a8f-af64-73fca8def346 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:14.173252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4e954-6f6e-4a8f-af64-73fca8def346 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:14.175958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff4e954-6f6e-4a8f-af64-73fca8def346 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-3ff4e954-6f6e-4a8f-af64-73fca8def346 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:14.222836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:14.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:14.225141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:14.225141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:14.225141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:14.225141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:14.234276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 602/1203] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:15:14.236563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7bbc2-127b-4dd7-9ce2-60805c77d08d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:14.467994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7bbc2-127b-4dd7-9ce2-60805c77d08d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:14.505306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:14.524615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18860b0e-8bd9-47da-961e-4e48ae042211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:14.526163 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860b0e-8bd9-47da-961e-4e48ae042211 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:14.532009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860b0e-8bd9-47da-961e-4e48ae042211 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:14.532988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18860b0e-8bd9-47da-961e-4e48ae042211 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:14.534762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18860b0e-8bd9-47da-961e-4e48ae042211 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:14.535983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18860b0e-8bd9-47da-961e-4e48ae042211 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:14.538367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18860b0e-8bd9-47da-961e-4e48ae042211 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:14.559976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 602/1204] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:14.566410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4e954-6f6e-4a8f-af64-73fca8def346 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:14.698377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:14.800107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff4e954-6f6e-4a8f-af64-73fca8def346 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:14.866693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ceb1f1-a1e0-4152-907b-2423b463e62a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:14.867988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ceb1f1-a1e0-4152-907b-2423b463e62a None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:14.868581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ceb1f1-a1e0-4152-907b-2423b463e62a None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:14.869069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ceb1f1-a1e0-4152-907b-2423b463e62a None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:14.870479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ceb1f1-a1e0-4152-907b-2423b463e62a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:14.873340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ceb1f1-a1e0-4152-907b-2423b463e62a None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:14.930796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 603/1205] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Jul 26 07:15:14 2022] POST /identity/v3/users => generated 251 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:15:17.654877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e16f48-76c1-4816-8782-7ecbc28ffd61 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:17.655519 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e16f48-76c1-4816-8782-7ecbc28ffd61 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:17.655613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e16f48-76c1-4816-8782-7ecbc28ffd61 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:17.657124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 603/1206] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:15:17.664043 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87983448-0ca7-48d7-bd77-5e1aa7176d7c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:17.664536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87983448-0ca7-48d7-bd77-5e1aa7176d7c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:17.664692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87983448-0ca7-48d7-bd77-5e1aa7176d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:17.685190 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-87983448-0ca7-48d7-bd77-5e1aa7176d7c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:17.686654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-87983448-0ca7-48d7-bd77-5e1aa7176d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:17.792093 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 604/1207] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:15:17.827000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e16f48-76c1-4816-8782-7ecbc28ffd61 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:17.989963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e16f48-76c1-4816-8782-7ecbc28ffd61 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:18.046511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15155a14-e742-4212-bc7d-f6493b2ac1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:18.047527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15155a14-e742-4212-bc7d-f6493b2ac1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:18.048025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15155a14-e742-4212-bc7d-f6493b2ac1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:18.048533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15155a14-e742-4212-bc7d-f6493b2ac1b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:18.068493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15155a14-e742-4212-bc7d-f6493b2ac1b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-15155a14-e742-4212-bc7d-f6493b2ac1b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:15:18.077588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:18.078733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:18.079654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 604/1208] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:18.086421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87983448-0ca7-48d7-bd77-5e1aa7176d7c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:18.136381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:18.231756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87983448-0ca7-48d7-bd77-5e1aa7176d7c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:18.295692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0259f-5bb6-4973-8ee7-47f7de0d4e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:18.296813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0259f-5bb6-4973-8ee7-47f7de0d4e22 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:18.296813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0259f-5bb6-4973-8ee7-47f7de0d4e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:18.297239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a0259f-5bb6-4973-8ee7-47f7de0d4e22 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:18.297759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0259f-5bb6-4973-8ee7-47f7de0d4e22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:18.298125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0259f-5bb6-4973-8ee7-47f7de0d4e22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:18.299711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a0259f-5bb6-4973-8ee7-47f7de0d4e22 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:18.310995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 605/1209] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:15:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:18.320635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15155a14-e742-4212-bc7d-f6493b2ac1b1 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:18.397504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:18.479206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15155a14-e742-4212-bc7d-f6493b2ac1b1 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:18.521878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d92be-972f-414e-9674-c7fb7547169f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:18.522622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d92be-972f-414e-9674-c7fb7547169f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:18.522878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d92be-972f-414e-9674-c7fb7547169f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:18.523157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4d92be-972f-414e-9674-c7fb7547169f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:18.540813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d92be-972f-414e-9674-c7fb7547169f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:18.545936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4d92be-972f-414e-9674-c7fb7547169f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-de4d92be-972f-414e-9674-c7fb7547169f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:18.572855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:18.574040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:18.575302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:18.575302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:18.575302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:18.575302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:18.577126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 605/1210] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:15:18.583338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0259f-5bb6-4973-8ee7-47f7de0d4e22 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:18.737726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:18.742299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a0259f-5bb6-4973-8ee7-47f7de0d4e22 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:18.797578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fd7fd-a4ed-4d19-b1f3-dc771c19266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:18.797578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fd7fd-a4ed-4d19-b1f3-dc771c19266e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:18.798173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fd7fd-a4ed-4d19-b1f3-dc771c19266e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:18.798370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4fd7fd-a4ed-4d19-b1f3-dc771c19266e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:18.799679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fd7fd-a4ed-4d19-b1f3-dc771c19266e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:18.800100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fd7fd-a4ed-4d19-b1f3-dc771c19266e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:18.802737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4fd7fd-a4ed-4d19-b1f3-dc771c19266e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:18.833276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 606/1211] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:18.834695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d92be-972f-414e-9674-c7fb7547169f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:19.013445 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4d92be-972f-414e-9674-c7fb7547169f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:19.015800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:19.072682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790853a-c9c2-42ea-ac64-15ea2659b0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:19.074454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790853a-c9c2-42ea-ac64-15ea2659b0bc None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:19.074454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790853a-c9c2-42ea-ac64-15ea2659b0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:19.074454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790853a-c9c2-42ea-ac64-15ea2659b0bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:19.087463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790853a-c9c2-42ea-ac64-15ea2659b0bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:19.089474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790853a-c9c2-42ea-ac64-15ea2659b0bc None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-8790853a-c9c2-42ea-ac64-15ea2659b0bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:15:19.107963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:19.109606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:19.110969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 606/1212] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:19.122071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fd7fd-a4ed-4d19-b1f3-dc771c19266e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:19.311260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4fd7fd-a4ed-4d19-b1f3-dc771c19266e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:19.362125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3f789-4267-4a35-b882-4cc0185da966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:19.362687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3f789-4267-4a35-b882-4cc0185da966 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:19.362967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3f789-4267-4a35-b882-4cc0185da966 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:19.363371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3f789-4267-4a35-b882-4cc0185da966 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:19.364230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3f789-4267-4a35-b882-4cc0185da966 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:19.364652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3f789-4267-4a35-b882-4cc0185da966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:19.366619 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b3f789-4267-4a35-b882-4cc0185da966 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:19.391861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 607/1213] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jul 26 07:15:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:19.399503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790853a-c9c2-42ea-ac64-15ea2659b0bc None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:19.555974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790853a-c9c2-42ea-ac64-15ea2659b0bc None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:19.617016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24394b5-8482-44d8-aae6-62773f900327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:19.617626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24394b5-8482-44d8-aae6-62773f900327 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:19.617952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24394b5-8482-44d8-aae6-62773f900327 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:19.618311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24394b5-8482-44d8-aae6-62773f900327 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:19.683567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:19.767125 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24394b5-8482-44d8-aae6-62773f900327 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:19.770105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24394b5-8482-44d8-aae6-62773f900327 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:19.975851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 607/1214] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:15:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:15:19.982625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3f789-4267-4a35-b882-4cc0185da966 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:20.008144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:20.116705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b3f789-4267-4a35-b882-4cc0185da966 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:20.191808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472de13-96b6-497d-b4bd-b82d55161e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:20.193048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472de13-96b6-497d-b4bd-b82d55161e1c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:20.193921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472de13-96b6-497d-b4bd-b82d55161e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:20.194477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5472de13-96b6-497d-b4bd-b82d55161e1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:20.212575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472de13-96b6-497d-b4bd-b82d55161e1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:20.214481 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5472de13-96b6-497d-b4bd-b82d55161e1c None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-5472de13-96b6-497d-b4bd-b82d55161e1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:20.239731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:20.241514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:20.243590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:20.243590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:20.243590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:20.243590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:20.243590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 608/1215] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:20.248000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24394b5-8482-44d8-aae6-62773f900327 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:20.346234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:20.403453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24394b5-8482-44d8-aae6-62773f900327 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:20.454683 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44cc72-1fd8-4dc8-a48f-2b9888e0ae35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:20.455331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44cc72-1fd8-4dc8-a48f-2b9888e0ae35 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:20.455614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44cc72-1fd8-4dc8-a48f-2b9888e0ae35 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:20.455894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44cc72-1fd8-4dc8-a48f-2b9888e0ae35 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:20.456619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44cc72-1fd8-4dc8-a48f-2b9888e0ae35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:20.456967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44cc72-1fd8-4dc8-a48f-2b9888e0ae35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:20.458591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44cc72-1fd8-4dc8-a48f-2b9888e0ae35 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:20.477779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 608/1216] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:20.482997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472de13-96b6-497d-b4bd-b82d55161e1c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:20.576876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:20.621981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5472de13-96b6-497d-b4bd-b82d55161e1c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:20.662526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418d4f8-8900-4e0b-8812-7d6c31ab2786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:20.662526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418d4f8-8900-4e0b-8812-7d6c31ab2786 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:20.662526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418d4f8-8900-4e0b-8812-7d6c31ab2786 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:20.662526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418d4f8-8900-4e0b-8812-7d6c31ab2786 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:20.669991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418d4f8-8900-4e0b-8812-7d6c31ab2786 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:20.671751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6418d4f8-8900-4e0b-8812-7d6c31ab2786 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6418d4f8-8900-4e0b-8812-7d6c31ab2786 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:15:20.685247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:20.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:20.687783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:15:20.687783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:15:20.687783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:15:20.687783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:15:20.687783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:15:20.687783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:15:20.687783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:20.687783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:20.687783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 609/1217] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:15:20.693477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44cc72-1fd8-4dc8-a48f-2b9888e0ae35 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:20.815986 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44cc72-1fd8-4dc8-a48f-2b9888e0ae35 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:20.846220 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:20.861673 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d0c71-b6ab-4afd-a098-5c3e47864f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:20.862461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d0c71-b6ab-4afd-a098-5c3e47864f3b None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:20.862755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d0c71-b6ab-4afd-a098-5c3e47864f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:20.863004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d0c71-b6ab-4afd-a098-5c3e47864f3b None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:20.863767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d0c71-b6ab-4afd-a098-5c3e47864f3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:20.864087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d0c71-b6ab-4afd-a098-5c3e47864f3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:20.865613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d0c71-b6ab-4afd-a098-5c3e47864f3b None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:20.881427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 609/1218] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:15:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:20.887827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418d4f8-8900-4e0b-8812-7d6c31ab2786 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:21.026581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6418d4f8-8900-4e0b-8812-7d6c31ab2786 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:21.064571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:21.070954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926df17-50e7-4b6f-a83c-8186dd24ef59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:21.071597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926df17-50e7-4b6f-a83c-8186dd24ef59 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:21.071892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926df17-50e7-4b6f-a83c-8186dd24ef59 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:21.072164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926df17-50e7-4b6f-a83c-8186dd24ef59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:21.072968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926df17-50e7-4b6f-a83c-8186dd24ef59 None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=51a5093a60ef45919ab6e159adf58336) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:21.073271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926df17-50e7-4b6f-a83c-8186dd24ef59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:21.074721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926df17-50e7-4b6f-a83c-8186dd24ef59 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:21.082321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 610/1219] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:15:20 2022] GET /identity/v3/role_assignments?user.id=51a5093a60ef45919ab6e159adf58336&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:23.171983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19df71-5517-409a-b743-235854739064 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:23.172934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19df71-5517-409a-b743-235854739064 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:23.172934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc19df71-5517-409a-b743-235854739064 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:23.174319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 610/1220] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:15:23.180489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad67c98-67c3-42ff-b9d1-d9742f0bcc15 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:23.181561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad67c98-67c3-42ff-b9d1-d9742f0bcc15 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:23.181561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad67c98-67c3-42ff-b9d1-d9742f0bcc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:23.201101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-aad67c98-67c3-42ff-b9d1-d9742f0bcc15 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:23.202348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-aad67c98-67c3-42ff-b9d1-d9742f0bcc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:23.285852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 611/1221] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:15:23.308087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19df71-5517-409a-b743-235854739064 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:23.451911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc19df71-5517-409a-b743-235854739064 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:23.504209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ead932-3338-458e-812b-2f074785251f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:23.504982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ead932-3338-458e-812b-2f074785251f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:23.505260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ead932-3338-458e-812b-2f074785251f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:23.505580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ead932-3338-458e-812b-2f074785251f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:23.516827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ead932-3338-458e-812b-2f074785251f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-48ead932-3338-458e-812b-2f074785251f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:15:23.521969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:23.523048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:15:23.524112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:23.526126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 611/1222] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:15:23.530147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad67c98-67c3-42ff-b9d1-d9742f0bcc15 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:23.654061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad67c98-67c3-42ff-b9d1-d9742f0bcc15 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:23.686494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:23.703737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d461b-76b6-4847-bfb3-99503d80dd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:23.704379 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d461b-76b6-4847-bfb3-99503d80dd81 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:23.704645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d461b-76b6-4847-bfb3-99503d80dd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:23.704924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8d461b-76b6-4847-bfb3-99503d80dd81 None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:23.705669 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d461b-76b6-4847-bfb3-99503d80dd81 None admin] RBAC: Adding query filter params (name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:23.706008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d461b-76b6-4847-bfb3-99503d80dd81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:23.707401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8d461b-76b6-4847-bfb3-99503d80dd81 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:23.724114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 612/1223] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:23.731045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ead932-3338-458e-812b-2f074785251f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:23.785510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:23.860858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ead932-3338-458e-812b-2f074785251f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:23.902062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6c50e-e214-4e65-a931-5d4e86a73ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:23.902784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6c50e-e214-4e65-a931-5d4e86a73ffa None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:23.903092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6c50e-e214-4e65-a931-5d4e86a73ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:23.903444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6c50e-e214-4e65-a931-5d4e86a73ffa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:23.915818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6c50e-e214-4e65-a931-5d4e86a73ffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:23.917429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6c50e-e214-4e65-a931-5d4e86a73ffa None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-d5b6c50e-e214-4e65-a931-5d4e86a73ffa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:23.933168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:23.935024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:23.936775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:23.936775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:23.936775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:23.936775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:23.936775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 612/1224] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 26 07:15:23.940334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d461b-76b6-4847-bfb3-99503d80dd81 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:24.058009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8d461b-76b6-4847-bfb3-99503d80dd81 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:24.097562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436d3eb-ccf2-408c-8954-5c1fce0c092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:24.098226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436d3eb-ccf2-408c-8954-5c1fce0c092e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:24.098509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436d3eb-ccf2-408c-8954-5c1fce0c092e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:24.098788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436d3eb-ccf2-408c-8954-5c1fce0c092e None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:24.099497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436d3eb-ccf2-408c-8954-5c1fce0c092e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:24.099802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436d3eb-ccf2-408c-8954-5c1fce0c092e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:24.101201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436d3eb-ccf2-408c-8954-5c1fce0c092e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:24.115148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 613/1225] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:24.121231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6c50e-e214-4e65-a931-5d4e86a73ffa None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:24.231072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:24.239486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6c50e-e214-4e65-a931-5d4e86a73ffa None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:24.278669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d391e3b-5fbf-4559-bf4f-446744cd7bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:24.279590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d391e3b-5fbf-4559-bf4f-446744cd7bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:24.279987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d391e3b-5fbf-4559-bf4f-446744cd7bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:24.280275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d391e3b-5fbf-4559-bf4f-446744cd7bc3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:24.289717 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d391e3b-5fbf-4559-bf4f-446744cd7bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:24.291314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d391e3b-5fbf-4559-bf4f-446744cd7bc3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-3d391e3b-5fbf-4559-bf4f-446744cd7bc3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:15:24.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:24.306401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:24.309088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 613/1226] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:24.311740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436d3eb-ccf2-408c-8954-5c1fce0c092e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:24.370235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:24.438068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436d3eb-ccf2-408c-8954-5c1fce0c092e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:24.483698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb02945-357c-4c09-9c1d-3aaad3795f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:24.484901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb02945-357c-4c09-9c1d-3aaad3795f08 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:24.485862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb02945-357c-4c09-9c1d-3aaad3795f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:24.486368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb02945-357c-4c09-9c1d-3aaad3795f08 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:24.487936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb02945-357c-4c09-9c1d-3aaad3795f08 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:24.488502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb02945-357c-4c09-9c1d-3aaad3795f08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:24.491442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb02945-357c-4c09-9c1d-3aaad3795f08 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:24.515008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 614/1227] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jul 26 07:15:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:24.521274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d391e3b-5fbf-4559-bf4f-446744cd7bc3 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:24.665778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d391e3b-5fbf-4559-bf4f-446744cd7bc3 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:24.719329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae37ea0-fa2c-4aec-b62b-08245740f2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:24.720240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae37ea0-fa2c-4aec-b62b-08245740f2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:24.720697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae37ea0-fa2c-4aec-b62b-08245740f2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:24.721132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae37ea0-fa2c-4aec-b62b-08245740f2c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:24.736909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae37ea0-fa2c-4aec-b62b-08245740f2c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:24.738998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae37ea0-fa2c-4aec-b62b-08245740f2c2 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5ae37ea0-fa2c-4aec-b62b-08245740f2c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:24.756949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:24.758137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:24.759072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:24.759072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:24.759072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:24.759072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:24.759072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 614/1228] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:24.764121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb02945-357c-4c09-9c1d-3aaad3795f08 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:24.907373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb02945-357c-4c09-9c1d-3aaad3795f08 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:24.910066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:24.957930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0ac5a-231d-4dfd-9242-a7f215591289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:24.958842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0ac5a-231d-4dfd-9242-a7f215591289 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:24.959331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0ac5a-231d-4dfd-9242-a7f215591289 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:24.959913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0ac5a-231d-4dfd-9242-a7f215591289 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:24.961319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0ac5a-231d-4dfd-9242-a7f215591289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:24.962018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0ac5a-231d-4dfd-9242-a7f215591289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:24.964907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0ac5a-231d-4dfd-9242-a7f215591289 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:24.983057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 615/1229] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:24.990434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae37ea0-fa2c-4aec-b62b-08245740f2c2 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:25.003793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:25.118821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae37ea0-fa2c-4aec-b62b-08245740f2c2 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:25.167683 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a95fc-c058-44ba-a546-cde808eda8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:25.168422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a95fc-c058-44ba-a546-cde808eda8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:25.168886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a95fc-c058-44ba-a546-cde808eda8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:25.169194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6a95fc-c058-44ba-a546-cde808eda8b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:25.179555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a95fc-c058-44ba-a546-cde808eda8b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:25.182656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6a95fc-c058-44ba-a546-cde808eda8b8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-ef6a95fc-c058-44ba-a546-cde808eda8b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:15:25.205865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:25.208881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:25.210971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:15:25.210971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:15:25.210971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:15:25.210971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:15:25.210971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:15:25.210971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:15:25.210971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:25.210971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:25.210971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 615/1230] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:15:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:25.217661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0ac5a-231d-4dfd-9242-a7f215591289 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:25.345162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0ac5a-231d-4dfd-9242-a7f215591289 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:25.392957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945ec91-fd2a-49a8-b0cd-359571f172bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:25.394079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945ec91-fd2a-49a8-b0cd-359571f172bd None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:25.394574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945ec91-fd2a-49a8-b0cd-359571f172bd None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:25.395089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945ec91-fd2a-49a8-b0cd-359571f172bd None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:25.396465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945ec91-fd2a-49a8-b0cd-359571f172bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:25.397013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945ec91-fd2a-49a8-b0cd-359571f172bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:25.399785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945ec91-fd2a-49a8-b0cd-359571f172bd None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:25.422870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 616/1231] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:15:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:25.430812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a95fc-c058-44ba-a546-cde808eda8b8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:25.513134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:25.568966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6a95fc-c058-44ba-a546-cde808eda8b8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:25.633265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e600c99-31de-46f2-bef8-dc3703832a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:25.634610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e600c99-31de-46f2-bef8-dc3703832a29 None admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:25.634955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e600c99-31de-46f2-bef8-dc3703832a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:25.635350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e600c99-31de-46f2-bef8-dc3703832a29 None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/51a5093a60ef45919ab6e159adf58336/roles/e998fc875b3b45fc8bcab8bff2f1dc5a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:25.659600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e600c99-31de-46f2-bef8-dc3703832a29 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=51a5093a60ef45919ab6e159adf58336, role_id=e998fc875b3b45fc8bcab8bff2f1dc5a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:25.661427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e600c99-31de-46f2-bef8-dc3703832a29 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:25.683589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 616/1232] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:15:25 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/51a5093a60ef45919ab6e159adf58336/roles/e998fc875b3b45fc8bcab8bff2f1dc5a => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:15:27.902254 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e827583-e139-4842-8737-cb43c57c7be8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:27.903191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e827583-e139-4842-8737-cb43c57c7be8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:27.903191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e827583-e139-4842-8737-cb43c57c7be8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:27.904367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 617/1233] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:15:27.910410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51582626-efa1-42e8-aca7-e5b0a1332755 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:27.911047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51582626-efa1-42e8-aca7-e5b0a1332755 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:27.911047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51582626-efa1-42e8-aca7-e5b0a1332755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:27.929471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-51582626-efa1-42e8-aca7-e5b0a1332755 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:27.930708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-51582626-efa1-42e8-aca7-e5b0a1332755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:28.019583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 617/1234] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:15:28.043390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e827583-e139-4842-8737-cb43c57c7be8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:28.079767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:28.192111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e827583-e139-4842-8737-cb43c57c7be8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:28.234016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e111213-62d1-4624-8dc7-fcba28763016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:28.234923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e111213-62d1-4624-8dc7-fcba28763016 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:28.235676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e111213-62d1-4624-8dc7-fcba28763016 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:28.236088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e111213-62d1-4624-8dc7-fcba28763016 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:28.247340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e111213-62d1-4624-8dc7-fcba28763016 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-7e111213-62d1-4624-8dc7-fcba28763016 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:15:28.252405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:28.253526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:28.259860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 618/1235] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Jul 26 07:15:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:28.260567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51582626-efa1-42e8-aca7-e5b0a1332755 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:28.316774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:28.401590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51582626-efa1-42e8-aca7-e5b0a1332755 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:28.466700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495c672-2618-4bfe-b42d-ec54df36eadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:28.467365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495c672-2618-4bfe-b42d-ec54df36eadb None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:28.467674 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495c672-2618-4bfe-b42d-ec54df36eadb None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:28.467989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495c672-2618-4bfe-b42d-ec54df36eadb None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:28.468835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495c672-2618-4bfe-b42d-ec54df36eadb None admin] RBAC: Adding query filter params (name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:28.469116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495c672-2618-4bfe-b42d-ec54df36eadb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:28.470646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c495c672-2618-4bfe-b42d-ec54df36eadb None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:28.481673 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 618/1236] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Jul 26 07:15:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:28.489231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e111213-62d1-4624-8dc7-fcba28763016 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:28.614335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:28.642064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e111213-62d1-4624-8dc7-fcba28763016 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:28.695886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60147b8b-2526-4143-bc1a-c6e2a716caa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:28.696463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60147b8b-2526-4143-bc1a-c6e2a716caa5 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:28.696746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60147b8b-2526-4143-bc1a-c6e2a716caa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:28.697065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60147b8b-2526-4143-bc1a-c6e2a716caa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:28.711257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60147b8b-2526-4143-bc1a-c6e2a716caa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:28.713832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60147b8b-2526-4143-bc1a-c6e2a716caa5 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-60147b8b-2526-4143-bc1a-c6e2a716caa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:28.736767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:28.738113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:28.739661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:28.739661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:28.739661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:28.739661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:28.739661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 619/1237] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:28.745966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495c672-2618-4bfe-b42d-ec54df36eadb None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:28.876488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495c672-2618-4bfe-b42d-ec54df36eadb None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:28.888929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:28.929116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5042-6f1a-47dc-8ca0-bc4cece00642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:28.930282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5042-6f1a-47dc-8ca0-bc4cece00642 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:28.930626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5042-6f1a-47dc-8ca0-bc4cece00642 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:28.930963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ad5042-6f1a-47dc-8ca0-bc4cece00642 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:28.931814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad5042-6f1a-47dc-8ca0-bc4cece00642 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:28.932233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad5042-6f1a-47dc-8ca0-bc4cece00642 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:28.934003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ad5042-6f1a-47dc-8ca0-bc4cece00642 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:28.950490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 619/1238] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:28.958576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60147b8b-2526-4143-bc1a-c6e2a716caa5 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:29.093549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60147b8b-2526-4143-bc1a-c6e2a716caa5 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:29.135419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29228e6-23b6-4eee-9396-2c3cb1cc30be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:29.136206 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29228e6-23b6-4eee-9396-2c3cb1cc30be None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:29.136491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29228e6-23b6-4eee-9396-2c3cb1cc30be None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:29.136931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29228e6-23b6-4eee-9396-2c3cb1cc30be None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:29.140631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:29.148996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29228e6-23b6-4eee-9396-2c3cb1cc30be None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:29.150799 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29228e6-23b6-4eee-9396-2c3cb1cc30be None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-e29228e6-23b6-4eee-9396-2c3cb1cc30be None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:15:29.170625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:29.178232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:29.179283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 620/1239] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:28 2022] GET /identity/v3/users/placement => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:29.180017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5042-6f1a-47dc-8ca0-bc4cece00642 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:29.311052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ad5042-6f1a-47dc-8ca0-bc4cece00642 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:29.358080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaadda-607a-4df2-86fd-f256388fed7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:29.358718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaadda-607a-4df2-86fd-f256388fed7d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:29.359113 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaadda-607a-4df2-86fd-f256388fed7d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:29.359298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffaadda-607a-4df2-86fd-f256388fed7d None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:29.360065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaadda-607a-4df2-86fd-f256388fed7d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:29.360749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaadda-607a-4df2-86fd-f256388fed7d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:29.362226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffaadda-607a-4df2-86fd-f256388fed7d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:29.383196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 620/1240] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jul 26 07:15:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:29.390239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29228e6-23b6-4eee-9396-2c3cb1cc30be None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:29.560205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29228e6-23b6-4eee-9396-2c3cb1cc30be None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:29.608719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfc835-f6a5-4ca3-9830-41e166d95c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:29.609682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc835-f6a5-4ca3-9830-41e166d95c58 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:29.610209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc835-f6a5-4ca3-9830-41e166d95c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:29.611052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdfc835-f6a5-4ca3-9830-41e166d95c58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:29.699471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:29.710215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfc835-f6a5-4ca3-9830-41e166d95c58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:29.711949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdfc835-f6a5-4ca3-9830-41e166d95c58 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:29.874475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 621/1241] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:15:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 26 07:15:29.880984 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaadda-607a-4df2-86fd-f256388fed7d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:29.915476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:30.036778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffaadda-607a-4df2-86fd-f256388fed7d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:30.092587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9de26-ff3a-4b60-a09a-12e19656038c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:30.093298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9de26-ff3a-4b60-a09a-12e19656038c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:30.093693 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9de26-ff3a-4b60-a09a-12e19656038c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:30.094040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9de26-ff3a-4b60-a09a-12e19656038c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:30.107760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9de26-ff3a-4b60-a09a-12e19656038c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:30.109499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9de26-ff3a-4b60-a09a-12e19656038c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-e0f9de26-ff3a-4b60-a09a-12e19656038c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:30.130477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:30.132556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:30.134348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:30.134348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:30.134348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:30.134348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:30.134348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 621/1242] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:30.139174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfc835-f6a5-4ca3-9830-41e166d95c58 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:30.291826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:30.300658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdfc835-f6a5-4ca3-9830-41e166d95c58 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:30.353641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cbf2a-c7e6-4ea8-9f2d-f34b10c7539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:30.355696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cbf2a-c7e6-4ea8-9f2d-f34b10c7539b None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:30.356246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cbf2a-c7e6-4ea8-9f2d-f34b10c7539b None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:30.356740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cbf2a-c7e6-4ea8-9f2d-f34b10c7539b None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:30.358050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cbf2a-c7e6-4ea8-9f2d-f34b10c7539b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:30.358596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cbf2a-c7e6-4ea8-9f2d-f34b10c7539b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:30.361457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cbf2a-c7e6-4ea8-9f2d-f34b10c7539b None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:30.383047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 622/1243] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:30.392692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9de26-ff3a-4b60-a09a-12e19656038c None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:30.498861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:30.533369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9de26-ff3a-4b60-a09a-12e19656038c None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:30.580123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf3369-e4e6-4e5a-86ec-abd4f2f930e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:30.581180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf3369-e4e6-4e5a-86ec-abd4f2f930e4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:30.581598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf3369-e4e6-4e5a-86ec-abd4f2f930e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:30.582181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf3369-e4e6-4e5a-86ec-abd4f2f930e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:30.596320 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf3369-e4e6-4e5a-86ec-abd4f2f930e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:30.599733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf3369-e4e6-4e5a-86ec-abd4f2f930e4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-a2cf3369-e4e6-4e5a-86ec-abd4f2f930e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:15:30.624356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:30.625649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:30.626833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:15:30.626833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:15:30.626833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:15:30.626833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:15:30.626833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:15:30.626833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:15:30.626833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:30.626833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:30.627451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 622/1244] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:15:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:30.633321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cbf2a-c7e6-4ea8-9f2d-f34b10c7539b None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:30.767845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cbf2a-c7e6-4ea8-9f2d-f34b10c7539b None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:30.805713 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869f369-6afa-496f-9337-cce7548d3f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:30.806449 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869f369-6afa-496f-9337-cce7548d3f83 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:30.806730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869f369-6afa-496f-9337-cce7548d3f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:30.807239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869f369-6afa-496f-9337-cce7548d3f83 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:30.808096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869f369-6afa-496f-9337-cce7548d3f83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:30.808411 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869f369-6afa-496f-9337-cce7548d3f83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:30.810944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869f369-6afa-496f-9337-cce7548d3f83 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:30.833109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 623/1245] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:15:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:30.840985 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf3369-e4e6-4e5a-86ec-abd4f2f930e4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:30.967406 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf3369-e4e6-4e5a-86ec-abd4f2f930e4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:31.014698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac9bfb1-28e9-4fd8-b061-a29f1f9773e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:31.015509 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9bfb1-28e9-4fd8-b061-a29f1f9773e3 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:31.015898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9bfb1-28e9-4fd8-b061-a29f1f9773e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:31.016254 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac9bfb1-28e9-4fd8-b061-a29f1f9773e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:31.017442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9bfb1-28e9-4fd8-b061-a29f1f9773e3 None admin] RBAC: Adding query filter params (role.id=e998fc875b3b45fc8bcab8bff2f1dc5a, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=51a5093a60ef45919ab6e159adf58336) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:31.017956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9bfb1-28e9-4fd8-b061-a29f1f9773e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:31.020488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac9bfb1-28e9-4fd8-b061-a29f1f9773e3 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:31.030136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 623/1246] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:15:30 2022] GET /identity/v3/role_assignments?user.id=51a5093a60ef45919ab6e159adf58336&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=e998fc875b3b45fc8bcab8bff2f1dc5a => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:33.256844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483a8c3-23ce-4146-a12a-2e2dbe949bff None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:33.257471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483a8c3-23ce-4146-a12a-2e2dbe949bff None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:33.257471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483a8c3-23ce-4146-a12a-2e2dbe949bff None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:33.258974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 624/1247] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:15:33.265471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98a39b-c9ff-4535-9f9e-54ad72846c85 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:33.266051 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98a39b-c9ff-4535-9f9e-54ad72846c85 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:33.266138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98a39b-c9ff-4535-9f9e-54ad72846c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:33.273701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:33.293473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4e98a39b-c9ff-4535-9f9e-54ad72846c85 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:33.295569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4e98a39b-c9ff-4535-9f9e-54ad72846c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:33.383724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 624/1248] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:15:33.407145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483a8c3-23ce-4146-a12a-2e2dbe949bff None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:33.428336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:33.526411 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4483a8c3-23ce-4146-a12a-2e2dbe949bff None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:33.567301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f4104-5a6e-4afe-98f3-b65c4b4a01da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:33.568059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4104-5a6e-4afe-98f3-b65c4b4a01da None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:33.568372 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4104-5a6e-4afe-98f3-b65c4b4a01da None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:33.568727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f4104-5a6e-4afe-98f3-b65c4b4a01da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:33.581183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f4104-5a6e-4afe-98f3-b65c4b4a01da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f83f4104-5a6e-4afe-98f3-b65c4b4a01da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:15:33.586704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:33.587822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:33.588777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 625/1249] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:15:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:33.594875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98a39b-c9ff-4535-9f9e-54ad72846c85 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:33.713782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98a39b-c9ff-4535-9f9e-54ad72846c85 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:33.752991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:33.760749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5732551-0fc2-4d63-8b75-4a3cd06c1246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:33.761396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5732551-0fc2-4d63-8b75-4a3cd06c1246 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:33.761676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5732551-0fc2-4d63-8b75-4a3cd06c1246 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:33.761969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5732551-0fc2-4d63-8b75-4a3cd06c1246 None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:33.762699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5732551-0fc2-4d63-8b75-4a3cd06c1246 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:33.763003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5732551-0fc2-4d63-8b75-4a3cd06c1246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:33.764509 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5732551-0fc2-4d63-8b75-4a3cd06c1246 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:33.776275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 625/1250] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:15:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:33.782631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f4104-5a6e-4afe-98f3-b65c4b4a01da None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:33.881588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:33.914887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f4104-5a6e-4afe-98f3-b65c4b4a01da None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:33.957570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4d04b-2d20-4964-b538-7c36e6e83c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:33.958370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4d04b-2d20-4964-b538-7c36e6e83c11 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:33.958659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4d04b-2d20-4964-b538-7c36e6e83c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:33.958937 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4d04b-2d20-4964-b538-7c36e6e83c11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:33.971565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4d04b-2d20-4964-b538-7c36e6e83c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:33.973036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a4d04b-2d20-4964-b538-7c36e6e83c11 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-44a4d04b-2d20-4964-b538-7c36e6e83c11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:33.990430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:33.991947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:33.992943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:33.992943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:33.992943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:33.992943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:33.992943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 626/1251] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:33.998509 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5732551-0fc2-4d63-8b75-4a3cd06c1246 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:34.120129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5732551-0fc2-4d63-8b75-4a3cd06c1246 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:34.171613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c54da-99a4-4c58-865a-dd4a503ffdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:34.172341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c54da-99a4-4c58-865a-dd4a503ffdde None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:34.172599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c54da-99a4-4c58-865a-dd4a503ffdde None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:34.172951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c54da-99a4-4c58-865a-dd4a503ffdde None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:34.173710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c54da-99a4-4c58-865a-dd4a503ffdde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:34.174050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c54da-99a4-4c58-865a-dd4a503ffdde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:34.175537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c54da-99a4-4c58-865a-dd4a503ffdde None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:34.195681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 626/1252] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:34.203555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4d04b-2d20-4964-b538-7c36e6e83c11 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:34.317623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4d04b-2d20-4964-b538-7c36e6e83c11 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:34.346085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:34.354809 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5eaa90-2343-4926-824c-da633b3b1dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:34.355533 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5eaa90-2343-4926-824c-da633b3b1dcf None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:34.355816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5eaa90-2343-4926-824c-da633b3b1dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:34.356112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5eaa90-2343-4926-824c-da633b3b1dcf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:34.366062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5eaa90-2343-4926-824c-da633b3b1dcf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:34.367526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5eaa90-2343-4926-824c-da633b3b1dcf None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-1b5eaa90-2343-4926-824c-da633b3b1dcf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:15:34.382632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:34.384583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:34.385846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 627/1253] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:34.391952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c54da-99a4-4c58-865a-dd4a503ffdde None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:34.472254 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:34.527694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c54da-99a4-4c58-865a-dd4a503ffdde None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:34.567113 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dffbef-5dde-4e60-9ed6-6df09ea3c0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:34.567828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dffbef-5dde-4e60-9ed6-6df09ea3c0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:34.568177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dffbef-5dde-4e60-9ed6-6df09ea3c0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:34.568452 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dffbef-5dde-4e60-9ed6-6df09ea3c0f8 None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:34.569233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dffbef-5dde-4e60-9ed6-6df09ea3c0f8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:34.569541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dffbef-5dde-4e60-9ed6-6df09ea3c0f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:34.571066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dffbef-5dde-4e60-9ed6-6df09ea3c0f8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:34.596631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 627/1254] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jul 26 07:15:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:34.597998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5eaa90-2343-4926-824c-da633b3b1dcf None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:34.717465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5eaa90-2343-4926-824c-da633b3b1dcf None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:34.757435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c081d5d-22f2-450d-94c5-534bfcbf04b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:34.758206 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c081d5d-22f2-450d-94c5-534bfcbf04b4 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:34.758680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c081d5d-22f2-450d-94c5-534bfcbf04b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:34.759027 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c081d5d-22f2-450d-94c5-534bfcbf04b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:34.828348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:34.848775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c081d5d-22f2-450d-94c5-534bfcbf04b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:34.850391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c081d5d-22f2-450d-94c5-534bfcbf04b4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:34.980431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 628/1255] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:15:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:15:34.988032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dffbef-5dde-4e60-9ed6-6df09ea3c0f8 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:35.120571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dffbef-5dde-4e60-9ed6-6df09ea3c0f8 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:35.136610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:35.170305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a5c2d-3e9b-471c-ba16-8b86091a323d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:35.170984 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a5c2d-3e9b-471c-ba16-8b86091a323d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:35.171351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a5c2d-3e9b-471c-ba16-8b86091a323d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:35.171624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a5c2d-3e9b-471c-ba16-8b86091a323d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:35.184624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a5c2d-3e9b-471c-ba16-8b86091a323d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:35.186495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a5c2d-3e9b-471c-ba16-8b86091a323d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-148a5c2d-3e9b-471c-ba16-8b86091a323d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:35.205870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:35.207082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:35.208366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:35.208366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:35.208366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:35.208366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:35.210632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 628/1256] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:35.218974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c081d5d-22f2-450d-94c5-534bfcbf04b4 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:35.326280 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:35.342036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c081d5d-22f2-450d-94c5-534bfcbf04b4 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:35.383447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c22c6-b41d-401a-a0f4-4207b07059c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:35.384396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c22c6-b41d-401a-a0f4-4207b07059c0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:35.384923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c22c6-b41d-401a-a0f4-4207b07059c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:35.385477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c22c6-b41d-401a-a0f4-4207b07059c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:35.386718 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c22c6-b41d-401a-a0f4-4207b07059c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:35.387294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c22c6-b41d-401a-a0f4-4207b07059c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:35.388964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c22c6-b41d-401a-a0f4-4207b07059c0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:35.401296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 629/1257] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:35.407152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a5c2d-3e9b-471c-ba16-8b86091a323d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:35.538993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a5c2d-3e9b-471c-ba16-8b86091a323d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:35.589625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b84d3-fdda-4bb7-89c5-c31ff21afac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:35.590281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b84d3-fdda-4bb7-89c5-c31ff21afac7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:35.590563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b84d3-fdda-4bb7-89c5-c31ff21afac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:35.590835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b84d3-fdda-4bb7-89c5-c31ff21afac7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:35.602600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b84d3-fdda-4bb7-89c5-c31ff21afac7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:35.605375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b84d3-fdda-4bb7-89c5-c31ff21afac7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6c2b84d3-fdda-4bb7-89c5-c31ff21afac7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:15:35.621118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:35.622323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:35.628612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:15:35.628612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:15:35.628612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:15:35.628612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:15:35.628612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:15:35.628612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:15:35.628612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:35.628612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:35.628612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 629/1258] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:35.630357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c22c6-b41d-401a-a0f4-4207b07059c0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:35.757480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c22c6-b41d-401a-a0f4-4207b07059c0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:35.796469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ff2de-dae8-4c81-b108-52ec7ada8cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:35.797199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff2de-dae8-4c81-b108-52ec7ada8cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:35.797463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff2de-dae8-4c81-b108-52ec7ada8cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:35.797903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ff2de-dae8-4c81-b108-52ec7ada8cf1 None admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:35.798613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ff2de-dae8-4c81-b108-52ec7ada8cf1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:35.798951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ff2de-dae8-4c81-b108-52ec7ada8cf1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:35.800528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ff2de-dae8-4c81-b108-52ec7ada8cf1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:35.817489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 630/1259] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:15:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:35.824323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b84d3-fdda-4bb7-89c5-c31ff21afac7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:35.855221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:35.940398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b84d3-fdda-4bb7-89c5-c31ff21afac7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:35.983110 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff2e94-5adc-4fdc-a8ae-3284212f4a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:35.984136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff2e94-5adc-4fdc-a8ae-3284212f4a43 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:35.984638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff2e94-5adc-4fdc-a8ae-3284212f4a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:35.985185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff2e94-5adc-4fdc-a8ae-3284212f4a43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:35.986843 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff2e94-5adc-4fdc-a8ae-3284212f4a43 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=51a5093a60ef45919ab6e159adf58336) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:35.987404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff2e94-5adc-4fdc-a8ae-3284212f4a43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:35.990198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff2e94-5adc-4fdc-a8ae-3284212f4a43 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:35.999903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 630/1260] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:15:35 2022] GET /identity/v3/role_assignments?user.id=51a5093a60ef45919ab6e159adf58336&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:38.027779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9706d5-c4a2-48c6-9936-6839cc033345 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:38.028347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9706d5-c4a2-48c6-9936-6839cc033345 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:38.028347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9706d5-c4a2-48c6-9936-6839cc033345 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:38.029683 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 631/1261] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:15:38.035348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb40bc-ee9b-4b8f-8f4b-9bb39031b68f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:38.035891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb40bc-ee9b-4b8f-8f4b-9bb39031b68f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:38.035891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb40bc-ee9b-4b8f-8f4b-9bb39031b68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:38.058109 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-50bb40bc-ee9b-4b8f-8f4b-9bb39031b68f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:38.059325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-50bb40bc-ee9b-4b8f-8f4b-9bb39031b68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:38.136897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 631/1262] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:15:38.162007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9706d5-c4a2-48c6-9936-6839cc033345 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:38.174392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:38.291848 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9706d5-c4a2-48c6-9936-6839cc033345 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:38.329017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c48a2-9b07-4f2b-8f02-4bf66e45b31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:38.329660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c48a2-9b07-4f2b-8f02-4bf66e45b31c None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:38.329967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c48a2-9b07-4f2b-8f02-4bf66e45b31c None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:38.330240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c48a2-9b07-4f2b-8f02-4bf66e45b31c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:38.341581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c48a2-9b07-4f2b-8f02-4bf66e45b31c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d29c48a2-9b07-4f2b-8f02-4bf66e45b31c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:15:38.346544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:38.347640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:38.348663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 632/1263] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:15:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:38.354142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb40bc-ee9b-4b8f-8f4b-9bb39031b68f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:38.373323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:38.459398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb40bc-ee9b-4b8f-8f4b-9bb39031b68f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:38.502548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc822f09-f11f-441e-8245-2731d01e27ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:38.503174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc822f09-f11f-441e-8245-2731d01e27ce None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:38.503451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc822f09-f11f-441e-8245-2731d01e27ce None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:38.503718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc822f09-f11f-441e-8245-2731d01e27ce None admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:38.504436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc822f09-f11f-441e-8245-2731d01e27ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:38.504736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc822f09-f11f-441e-8245-2731d01e27ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:38.506155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc822f09-f11f-441e-8245-2731d01e27ce None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:38.515201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 632/1264] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:15:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:38.521501 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c48a2-9b07-4f2b-8f02-4bf66e45b31c None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:38.590626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:38.626276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c48a2-9b07-4f2b-8f02-4bf66e45b31c None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:38.661007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418e7d8-2352-475e-8c61-f481ec8a49ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:38.661698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418e7d8-2352-475e-8c61-f481ec8a49ce None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:38.662015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418e7d8-2352-475e-8c61-f481ec8a49ce None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:38.662288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418e7d8-2352-475e-8c61-f481ec8a49ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:38.673395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418e7d8-2352-475e-8c61-f481ec8a49ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:38.674890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418e7d8-2352-475e-8c61-f481ec8a49ce None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-c418e7d8-2352-475e-8c61-f481ec8a49ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:38.689569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:38.690690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:38.691710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:38.691710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:38.691710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:38.691710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:38.691710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 633/1265] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:38.696915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc822f09-f11f-441e-8245-2731d01e27ce None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:38.780679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:38.801368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc822f09-f11f-441e-8245-2731d01e27ce None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:38.835641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfafe1-45be-4c0f-8c9b-c0d440ca2c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:38.836266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfafe1-45be-4c0f-8c9b-c0d440ca2c12 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:38.836541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfafe1-45be-4c0f-8c9b-c0d440ca2c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:38.836809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfafe1-45be-4c0f-8c9b-c0d440ca2c12 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:38.837491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfafe1-45be-4c0f-8c9b-c0d440ca2c12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:38.837822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfafe1-45be-4c0f-8c9b-c0d440ca2c12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:38.839181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbfafe1-45be-4c0f-8c9b-c0d440ca2c12 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:38.850933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 633/1266] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:38.857284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418e7d8-2352-475e-8c61-f481ec8a49ce None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:38.960045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418e7d8-2352-475e-8c61-f481ec8a49ce None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:38.980403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:38.996455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12d910-a29b-44a3-a16a-29d1be026d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:38.997098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12d910-a29b-44a3-a16a-29d1be026d89 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:38.997371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12d910-a29b-44a3-a16a-29d1be026d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:38.997636 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a12d910-a29b-44a3-a16a-29d1be026d89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:39.005435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12d910-a29b-44a3-a16a-29d1be026d89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:39.006910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a12d910-a29b-44a3-a16a-29d1be026d89 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-6a12d910-a29b-44a3-a16a-29d1be026d89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:15:39.018492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:39.019666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:39.020839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:39.020839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:39.020839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:15:39.020839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:15:39.020839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:15:39.020839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:15:39.020839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:15:39.020839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:15:39.020839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:39.020839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:39.021905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 634/1267] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:39.026093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfafe1-45be-4c0f-8c9b-c0d440ca2c12 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:39.129378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfafe1-45be-4c0f-8c9b-c0d440ca2c12 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:39.169949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72730f64-fa4c-47a4-95b4-a665ae83549f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:39.170737 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72730f64-fa4c-47a4-95b4-a665ae83549f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:39.171020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72730f64-fa4c-47a4-95b4-a665ae83549f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:39.171495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72730f64-fa4c-47a4-95b4-a665ae83549f None admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:39.172527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72730f64-fa4c-47a4-95b4-a665ae83549f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:39.172973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72730f64-fa4c-47a4-95b4-a665ae83549f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:39.174730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72730f64-fa4c-47a4-95b4-a665ae83549f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:39.187118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:39.197618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 634/1268] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jul 26 07:15:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:39.204314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12d910-a29b-44a3-a16a-29d1be026d89 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:39.312117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a12d910-a29b-44a3-a16a-29d1be026d89 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:39.346743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea7c86-fa92-4822-85a0-8cec8f420891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:39.347387 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea7c86-fa92-4822-85a0-8cec8f420891 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:39.347666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea7c86-fa92-4822-85a0-8cec8f420891 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:39.347965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea7c86-fa92-4822-85a0-8cec8f420891 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:39.363990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea7c86-fa92-4822-85a0-8cec8f420891 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:39.365501 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea7c86-fa92-4822-85a0-8cec8f420891 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-d2ea7c86-fa92-4822-85a0-8cec8f420891 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:39.381253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:39.382389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:39.383284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:39.383284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:39.383284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:39.383284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:39.383284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 635/1269] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:39.388978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72730f64-fa4c-47a4-95b4-a665ae83549f None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:39.500059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72730f64-fa4c-47a4-95b4-a665ae83549f None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:39.541337 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b195766-6cd1-427d-a49e-28c85c7a8d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:39.541992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195766-6cd1-427d-a49e-28c85c7a8d05 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:39.542272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195766-6cd1-427d-a49e-28c85c7a8d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:39.542547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b195766-6cd1-427d-a49e-28c85c7a8d05 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:39.543222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b195766-6cd1-427d-a49e-28c85c7a8d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:39.543522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b195766-6cd1-427d-a49e-28c85c7a8d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:39.544938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b195766-6cd1-427d-a49e-28c85c7a8d05 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:39.556308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 635/1270] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:39.562723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea7c86-fa92-4822-85a0-8cec8f420891 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:39.598764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:39.671208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea7c86-fa92-4822-85a0-8cec8f420891 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:39.706216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b82ee-c696-40c6-8728-b936b534b26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:39.706862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b82ee-c696-40c6-8728-b936b534b26e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:39.707147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b82ee-c696-40c6-8728-b936b534b26e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:39.707419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b82ee-c696-40c6-8728-b936b534b26e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:39.714514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b82ee-c696-40c6-8728-b936b534b26e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:39.715973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b82ee-c696-40c6-8728-b936b534b26e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-8a3b82ee-c696-40c6-8728-b936b534b26e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:15:39.728833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:39.729925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:39.730986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:15:39.730986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:15:39.730986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:15:39.730986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:15:39.730986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:15:39.730986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:15:39.730986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:39.730986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:39.730986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 636/1271] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:39.736586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b195766-6cd1-427d-a49e-28c85c7a8d05 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:39.796533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:39.839722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b195766-6cd1-427d-a49e-28c85c7a8d05 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:39.875302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d26ce-2107-4cef-b67f-dd2b896580d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:39.875925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d26ce-2107-4cef-b67f-dd2b896580d1 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:39.876209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d26ce-2107-4cef-b67f-dd2b896580d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:39.876483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d26ce-2107-4cef-b67f-dd2b896580d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:39.877233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d26ce-2107-4cef-b67f-dd2b896580d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:39.877537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d26ce-2107-4cef-b67f-dd2b896580d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:39.879017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d26ce-2107-4cef-b67f-dd2b896580d1 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:39.891401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 636/1272] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:39.898117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b82ee-c696-40c6-8728-b936b534b26e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:39.991830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:40.002160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b82ee-c696-40c6-8728-b936b534b26e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:40.037178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67ed33-cefa-4aa0-adb5-aba8821ccf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:40.037872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67ed33-cefa-4aa0-adb5-aba8821ccf04 None admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:40.038151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67ed33-cefa-4aa0-adb5-aba8821ccf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:40.038425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67ed33-cefa-4aa0-adb5-aba8821ccf04 None admin] PATH_INFO: `/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/51a5093a60ef45919ab6e159adf58336/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:40.040349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67ed33-cefa-4aa0-adb5-aba8821ccf04 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e283157691d420caa8d7a3a389eadf7, user_id=51a5093a60ef45919ab6e159adf58336, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:40.041804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67ed33-cefa-4aa0-adb5-aba8821ccf04 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:40.056361 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 637/1273] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Jul 26 07:15:39 2022] PUT /identity/v3/projects/1e283157691d420caa8d7a3a389eadf7/users/51a5093a60ef45919ab6e159adf58336/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:15:42.026504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0fd94-8fd2-4d09-83d6-e5ec27dc87be None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:42.027336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0fd94-8fd2-4d09-83d6-e5ec27dc87be None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:42.027492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0fd94-8fd2-4d09-83d6-e5ec27dc87be None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:42.028856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 637/1274] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:15:42.034654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68339a11-ea99-4824-afde-c2b5f6e7a107 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:42.035068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68339a11-ea99-4824-afde-c2b5f6e7a107 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:42.035205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68339a11-ea99-4824-afde-c2b5f6e7a107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:42.054418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-68339a11-ea99-4824-afde-c2b5f6e7a107 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:42.055622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-68339a11-ea99-4824-afde-c2b5f6e7a107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:42.129275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 638/1275] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:15:42.154594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0fd94-8fd2-4d09-83d6-e5ec27dc87be None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:42.285112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0fd94-8fd2-4d09-83d6-e5ec27dc87be None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:42.302703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:42.321016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a848a4-652e-41fb-a115-6c1edd5b4b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:42.321661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a848a4-652e-41fb-a115-6c1edd5b4b8a None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:42.321974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a848a4-652e-41fb-a115-6c1edd5b4b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:42.322240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a848a4-652e-41fb-a115-6c1edd5b4b8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:42.332354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a848a4-652e-41fb-a115-6c1edd5b4b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-f3a848a4-652e-41fb-a115-6c1edd5b4b8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 26 07:15:42.337280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:42.338427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:42.339467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 638/1276] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Jul 26 07:15:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:42.344756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68339a11-ea99-4824-afde-c2b5f6e7a107 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:42.438165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:42.456715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68339a11-ea99-4824-afde-c2b5f6e7a107 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:42.495436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ba114-5ead-4266-8208-49dffe97600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:42.496056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba114-5ead-4266-8208-49dffe97600d None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:42.496337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba114-5ead-4266-8208-49dffe97600d None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:42.496609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409ba114-5ead-4266-8208-49dffe97600d None admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:42.497332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ba114-5ead-4266-8208-49dffe97600d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:42.497638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ba114-5ead-4266-8208-49dffe97600d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:42.499050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409ba114-5ead-4266-8208-49dffe97600d None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:42.507915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 639/1277] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Jul 26 07:15:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:42.514168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a848a4-652e-41fb-a115-6c1edd5b4b8a None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:42.631749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a848a4-652e-41fb-a115-6c1edd5b4b8a None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:42.669119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478275b-5832-4c3d-8d4d-13909cac77d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:42.669782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478275b-5832-4c3d-8d4d-13909cac77d6 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:42.670075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478275b-5832-4c3d-8d4d-13909cac77d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:42.670346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c478275b-5832-4c3d-8d4d-13909cac77d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:42.681298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478275b-5832-4c3d-8d4d-13909cac77d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:42.682794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c478275b-5832-4c3d-8d4d-13909cac77d6 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-c478275b-5832-4c3d-8d4d-13909cac77d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:42.697351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:42.698493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:42.699605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:42.699605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:42.699605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:42.699605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:42.699605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 639/1278] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:42.704578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ba114-5ead-4266-8208-49dffe97600d None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:42.807973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409ba114-5ead-4266-8208-49dffe97600d None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:42.842355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389a80e-9e28-4362-a56a-d77997e504d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:42.842968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389a80e-9e28-4362-a56a-d77997e504d0 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:42.843245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389a80e-9e28-4362-a56a-d77997e504d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:42.843517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389a80e-9e28-4362-a56a-d77997e504d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:42.844203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389a80e-9e28-4362-a56a-d77997e504d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:42.844504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389a80e-9e28-4362-a56a-d77997e504d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:42.845886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389a80e-9e28-4362-a56a-d77997e504d0 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:42.857895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 640/1279] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:42.864140 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478275b-5832-4c3d-8d4d-13909cac77d6 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:42.898032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:42.969128 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c478275b-5832-4c3d-8d4d-13909cac77d6 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:43.004354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dbb3b-c795-40a5-955a-49f7511ff6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:43.005026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dbb3b-c795-40a5-955a-49f7511ff6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:43.005301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dbb3b-c795-40a5-955a-49f7511ff6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:43.005567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7dbb3b-c795-40a5-955a-49f7511ff6c4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:43.013319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7dbb3b-c795-40a5-955a-49f7511ff6c4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:43.014787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7dbb3b-c795-40a5-955a-49f7511ff6c4 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-5f7dbb3b-c795-40a5-955a-49f7511ff6c4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 26 07:15:43.027885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:43.029460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:43.030727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:43.030727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:43.030727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 26 07:15:43.030727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 26 07:15:43.030727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 26 07:15:43.030727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 26 07:15:43.030727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 26 07:15:43.030727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 26 07:15:43.030727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 26 07:15:43.030727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:43.033824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 640/1280] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:43.035009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389a80e-9e28-4362-a56a-d77997e504d0 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:43.039175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:43.141551 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389a80e-9e28-4362-a56a-d77997e504d0 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:43.180354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe70ed4-8020-4694-be2c-c2e134b201f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:43.180908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe70ed4-8020-4694-be2c-c2e134b201f9 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:43.181193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe70ed4-8020-4694-be2c-c2e134b201f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:43.181467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe70ed4-8020-4694-be2c-c2e134b201f9 None admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:43.182316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe70ed4-8020-4694-be2c-c2e134b201f9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:43.182627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe70ed4-8020-4694-be2c-c2e134b201f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:43.184067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe70ed4-8020-4694-be2c-c2e134b201f9 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:43.202626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 641/1281] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Jul 26 07:15:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:15:43.208862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dbb3b-c795-40a5-955a-49f7511ff6c4 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:43.303451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:43.314094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7dbb3b-c795-40a5-955a-49f7511ff6c4 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:43.349054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53536c-6a82-40f5-b901-1491d84350cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:43.349678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53536c-6a82-40f5-b901-1491d84350cd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:43.349993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53536c-6a82-40f5-b901-1491d84350cd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:43.350265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f53536c-6a82-40f5-b901-1491d84350cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:43.418697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53536c-6a82-40f5-b901-1491d84350cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:43.420148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f53536c-6a82-40f5-b901-1491d84350cd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:43.511156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:43.523562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 641/1282] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Jul 26 07:15:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 26 07:15:43.529585 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe70ed4-8020-4694-be2c-c2e134b201f9 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:43.602786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:43.641445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe70ed4-8020-4694-be2c-c2e134b201f9 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:43.676369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257c40c-bf7a-4f08-b7a0-fa4f6202b751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:43.677122 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257c40c-bf7a-4f08-b7a0-fa4f6202b751 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:43.677412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257c40c-bf7a-4f08-b7a0-fa4f6202b751 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:43.677673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0257c40c-bf7a-4f08-b7a0-fa4f6202b751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:43.688741 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257c40c-bf7a-4f08-b7a0-fa4f6202b751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:43.690291 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0257c40c-bf7a-4f08-b7a0-fa4f6202b751 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-0257c40c-bf7a-4f08-b7a0-fa4f6202b751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 26 07:15:43.704817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:43.705867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:43.706974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 26 07:15:43.706974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 26 07:15:43.706974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 26 07:15:43.706974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:43.706974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 642/1283] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:43.712142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53536c-6a82-40f5-b901-1491d84350cd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:43.814866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f53536c-6a82-40f5-b901-1491d84350cd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:43.849871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc4b3d-c8c3-4029-b071-c574257b0c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:43.850499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4b3d-c8c3-4029-b071-c574257b0c56 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:43.850785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4b3d-c8c3-4029-b071-c574257b0c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:43.851060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bc4b3d-c8c3-4029-b071-c574257b0c56 None admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:43.851759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc4b3d-c8c3-4029-b071-c574257b0c56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:43.852062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc4b3d-c8c3-4029-b071-c574257b0c56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:43.853605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bc4b3d-c8c3-4029-b071-c574257b0c56 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:43.865319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 642/1284] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Jul 26 07:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:43.871427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257c40c-bf7a-4f08-b7a0-fa4f6202b751 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:43.978451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0257c40c-bf7a-4f08-b7a0-fa4f6202b751 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:43.992450 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:44.013838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee61d2-f417-4545-ab04-a9331ff7382e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:44.014481 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee61d2-f417-4545-ab04-a9331ff7382e None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:44.014757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee61d2-f417-4545-ab04-a9331ff7382e None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:44.015029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ee61d2-f417-4545-ab04-a9331ff7382e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:44.021926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee61d2-f417-4545-ab04-a9331ff7382e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:44.023349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ee61d2-f417-4545-ab04-a9331ff7382e None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application [None req-f9ee61d2-f417-4545-ab04-a9331ff7382e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 26 07:15:44.034710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:44.036286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:44.037344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 07:15:44.037344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 07:15:44.037344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 07:15:44.037344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 07:15:44.037344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 07:15:44.037344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 07:15:44.037344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 26 07:15:44.037344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: ERROR keystone.server.flask.application Jul 26 07:15:44.037344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 643/1285] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Jul 26 07:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:44.042128 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc4b3d-c8c3-4029-b071-c574257b0c56 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:44.092237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:44.144427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bc4b3d-c8c3-4029-b071-c574257b0c56 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:44.180739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ade069-f4ee-44a5-bafd-4596ad81c01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:44.181361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ade069-f4ee-44a5-bafd-4596ad81c01f None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:44.181641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ade069-f4ee-44a5-bafd-4596ad81c01f None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:44.181942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ade069-f4ee-44a5-bafd-4596ad81c01f None admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:44.182685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ade069-f4ee-44a5-bafd-4596ad81c01f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:44.182987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ade069-f4ee-44a5-bafd-4596ad81c01f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:44.184419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ade069-f4ee-44a5-bafd-4596ad81c01f None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:44.196800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 643/1286] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Jul 26 07:15:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:44.203341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee61d2-f417-4545-ab04-a9331ff7382e None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:44.323710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ee61d2-f417-4545-ab04-a9331ff7382e None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:44.359049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5ba1cf-007e-4058-b45d-9e1ab2b84220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:44.359661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ba1cf-007e-4058-b45d-9e1ab2b84220 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:44.359940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ba1cf-007e-4058-b45d-9e1ab2b84220 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:44.360170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5ba1cf-007e-4058-b45d-9e1ab2b84220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:44.361356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ba1cf-007e-4058-b45d-9e1ab2b84220 None admin] RBAC: Adding query filter params (role.id=c0cd7b1474694ed7989af245c9f34118, scope.project.id=1e283157691d420caa8d7a3a389eadf7, user.id=51a5093a60ef45919ab6e159adf58336) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:44.361665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ba1cf-007e-4058-b45d-9e1ab2b84220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:44.363120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5ba1cf-007e-4058-b45d-9e1ab2b84220 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:44.369108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 644/1287] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Jul 26 07:15:44 2022] GET /identity/v3/role_assignments?user.id=51a5093a60ef45919ab6e159adf58336&scope.project.id=1e283157691d420caa8d7a3a389eadf7&role.id=c0cd7b1474694ed7989af245c9f34118 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:46.379623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c11b1-28ef-4f7a-8c0d-dd0441bd5a0e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:46.380234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c11b1-28ef-4f7a-8c0d-dd0441bd5a0e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:46.380234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937c11b1-28ef-4f7a-8c0d-dd0441bd5a0e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:46.381591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 644/1288] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:15:46.386936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb29246-6bf5-4b86-a25c-088107e8f593 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:46.387379 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb29246-6bf5-4b86-a25c-088107e8f593 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:46.387471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb29246-6bf5-4b86-a25c-088107e8f593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:46.406872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4bb29246-6bf5-4b86-a25c-088107e8f593 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:46.408092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4bb29246-6bf5-4b86-a25c-088107e8f593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:46.439777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:46.480649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 645/1289] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:15:46.502601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c11b1-28ef-4f7a-8c0d-dd0441bd5a0e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:46.606603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937c11b1-28ef-4f7a-8c0d-dd0441bd5a0e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:46.617921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:46.644196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e952781-a0c4-4f28-b1d5-dc3e3a5e5c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:46.644759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e952781-a0c4-4f28-b1d5-dc3e3a5e5c8d None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:46.645035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e952781-a0c4-4f28-b1d5-dc3e3a5e5c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:46.645307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e952781-a0c4-4f28-b1d5-dc3e3a5e5c8d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:46.646031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e952781-a0c4-4f28-b1d5-dc3e3a5e5c8d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:46.647398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e952781-a0c4-4f28-b1d5-dc3e3a5e5c8d None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-1e952781-a0c4-4f28-b1d5-dc3e3a5e5c8d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:46.656754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:46.658041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:46.658949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 645/1290] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jul 26 07:15:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:46.664324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb29246-6bf5-4b86-a25c-088107e8f593 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:46.768230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb29246-6bf5-4b86-a25c-088107e8f593 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:46.805760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdcc154-8339-44f4-a1dd-421d82b3eea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:46.806388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc154-8339-44f4-a1dd-421d82b3eea1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:46.806661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc154-8339-44f4-a1dd-421d82b3eea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:46.806922 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdcc154-8339-44f4-a1dd-421d82b3eea1 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:46.807594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcc154-8339-44f4-a1dd-421d82b3eea1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:46.807890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcc154-8339-44f4-a1dd-421d82b3eea1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:46.809250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdcc154-8339-44f4-a1dd-421d82b3eea1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:46.814495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 646/1291] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Jul 26 07:15:46 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:46.820859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e952781-a0c4-4f28-b1d5-dc3e3a5e5c8d None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:46.926512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e952781-a0c4-4f28-b1d5-dc3e3a5e5c8d None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:46.962097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35371014-da8b-4f6c-a049-c29873f7fb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:46.962725 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35371014-da8b-4f6c-a049-c29873f7fb3c None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:46.963004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35371014-da8b-4f6c-a049-c29873f7fb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:46.963271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35371014-da8b-4f6c-a049-c29873f7fb3c None admin] PATH_INFO: `/v3/services` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:46.963946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35371014-da8b-4f6c-a049-c29873f7fb3c None admin] RBAC: Adding query filter params (type=placement) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:46.964248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35371014-da8b-4f6c-a049-c29873f7fb3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:46.965652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35371014-da8b-4f6c-a049-c29873f7fb3c None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:46.970700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 646/1292] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Jul 26 07:15:46 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:48.952939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70231e94-52e2-4cfb-af38-7bb1636db921 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:48.953572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70231e94-52e2-4cfb-af38-7bb1636db921 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:48.953572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70231e94-52e2-4cfb-af38-7bb1636db921 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:48.954876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 647/1293] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:15:48.960295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8054e-6869-479e-be23-f443a3f9f370 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:48.960750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8054e-6869-479e-be23-f443a3f9f370 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:48.960847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8054e-6869-479e-be23-f443a3f9f370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:48.979380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ebe8054e-6869-479e-be23-f443a3f9f370 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:48.980640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ebe8054e-6869-479e-be23-f443a3f9f370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:49.012176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:49.053296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 647/1294] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:15:49.075462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70231e94-52e2-4cfb-af38-7bb1636db921 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:49.108782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:49.189121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70231e94-52e2-4cfb-af38-7bb1636db921 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:49.230195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dd1480-25be-4ae3-a193-23dcac9b9aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:49.230739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1480-25be-4ae3-a193-23dcac9b9aa4 None admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:49.231019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1480-25be-4ae3-a193-23dcac9b9aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:49.231288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dd1480-25be-4ae3-a193-23dcac9b9aa4 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:49.231967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd1480-25be-4ae3-a193-23dcac9b9aa4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:49.233344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dd1480-25be-4ae3-a193-23dcac9b9aa4 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:49.249785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 648/1295] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Jul 26 07:15:49 2022] POST /identity/v3/services => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:15:51.213501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e347ce-3cb2-43f5-b0d8-99ece6836ba1 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:51.214138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e347ce-3cb2-43f5-b0d8-99ece6836ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:51.214138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e347ce-3cb2-43f5-b0d8-99ece6836ba1 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:51.215425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 648/1296] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:15:51.220708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bd9a5-74b7-42cf-b6c0-35c4cf4d4f5f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:51.221180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bd9a5-74b7-42cf-b6c0-35c4cf4d4f5f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:51.221288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bd9a5-74b7-42cf-b6c0-35c4cf4d4f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:51.240709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-474bd9a5-74b7-42cf-b6c0-35c4cf4d4f5f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:51.241951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-474bd9a5-74b7-42cf-b6c0-35c4cf4d4f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:51.318487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 649/1297] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:51 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 26 07:15:51.340999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e347ce-3cb2-43f5-b0d8-99ece6836ba1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:51.449805 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e347ce-3cb2-43f5-b0d8-99ece6836ba1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:51.483439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a30aa-229a-45ac-88f5-c3d743b42dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:51.483990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a30aa-229a-45ac-88f5-c3d743b42dfd None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:51.484265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a30aa-229a-45ac-88f5-c3d743b42dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:51.484578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a30aa-229a-45ac-88f5-c3d743b42dfd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:51.485154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a30aa-229a-45ac-88f5-c3d743b42dfd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:51.486915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168a30aa-229a-45ac-88f5-c3d743b42dfd None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-168a30aa-229a-45ac-88f5-c3d743b42dfd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:51.495814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:51.497027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:51.498160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 649/1298] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jul 26 07:15:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:51.503390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bd9a5-74b7-42cf-b6c0-35c4cf4d4f5f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:51.522598 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:51.605170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474bd9a5-74b7-42cf-b6c0-35c4cf4d4f5f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:51.639614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d296ebe-bb4b-455c-8911-6f69552a10d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:51.640224 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d296ebe-bb4b-455c-8911-6f69552a10d1 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:51.640502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d296ebe-bb4b-455c-8911-6f69552a10d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:51.640772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d296ebe-bb4b-455c-8911-6f69552a10d1 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:51.641448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d296ebe-bb4b-455c-8911-6f69552a10d1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:51.641792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d296ebe-bb4b-455c-8911-6f69552a10d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:51.643128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d296ebe-bb4b-455c-8911-6f69552a10d1 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:51.648383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 650/1299] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Jul 26 07:15:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:51.654771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a30aa-229a-45ac-88f5-c3d743b42dfd None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:51.687058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:51.761677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168a30aa-229a-45ac-88f5-c3d743b42dfd None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:51.796500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef85d2e-b392-4add-9ac0-5d8cd97b90f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:51.797124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef85d2e-b392-4add-9ac0-5d8cd97b90f8 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:51.797403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef85d2e-b392-4add-9ac0-5d8cd97b90f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:51.797681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef85d2e-b392-4add-9ac0-5d8cd97b90f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:51.798479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef85d2e-b392-4add-9ac0-5d8cd97b90f8 None admin] RBAC: Adding query filter params (interface=public, service_id=764c48374c1c4e85814bd00f1e35f1aa, region_id=RegionOne) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:51.798789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef85d2e-b392-4add-9ac0-5d8cd97b90f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:51.800176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef85d2e-b392-4add-9ac0-5d8cd97b90f8 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:51.805490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 650/1300] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Jul 26 07:15:51 2022] GET /identity/v3/endpoints?service_id=764c48374c1c4e85814bd00f1e35f1aa&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:51.811647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d296ebe-bb4b-455c-8911-6f69552a10d1 None admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:51.899915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:51.925530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d296ebe-bb4b-455c-8911-6f69552a10d1 None admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:51.963386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452251d4-e1b2-45b9-80aa-dd48d0358ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:51.963754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452251d4-e1b2-45b9-80aa-dd48d0358ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:51.964034 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452251d4-e1b2-45b9-80aa-dd48d0358ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:51.964308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452251d4-e1b2-45b9-80aa-dd48d0358ad2 None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:51.964964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452251d4-e1b2-45b9-80aa-dd48d0358ad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:51.966383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452251d4-e1b2-45b9-80aa-dd48d0358ad2 None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:51.973353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 651/1301] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Jul 26 07:15:51 2022] GET /identity/v3/services => generated 2191 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:15:53.940777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66613dde-fae9-44aa-b1ae-ae759d8e3c3b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:53.941378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66613dde-fae9-44aa-b1ae-ae759d8e3c3b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:53.941378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66613dde-fae9-44aa-b1ae-ae759d8e3c3b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:53.942794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 651/1302] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:15:53.948114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cace891-71fc-4d2a-8cc0-334258f6c5d4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:53.948545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cace891-71fc-4d2a-8cc0-334258f6c5d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:53.948643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cace891-71fc-4d2a-8cc0-334258f6c5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:53.967498 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2cace891-71fc-4d2a-8cc0-334258f6c5d4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:15:53.968713 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2cace891-71fc-4d2a-8cc0-334258f6c5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:15:54.039754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 652/1303] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:15:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 26 07:15:54.062020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66613dde-fae9-44aa-b1ae-ae759d8e3c3b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:54.160334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:54.167817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66613dde-fae9-44aa-b1ae-ae759d8e3c3b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:54.206078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716e562-ff5f-48e4-8d07-c21bb3a558f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:54.206882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716e562-ff5f-48e4-8d07-c21bb3a558f7 None admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:54.207296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716e562-ff5f-48e4-8d07-c21bb3a558f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:54.207701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716e562-ff5f-48e4-8d07-c21bb3a558f7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:54.208567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716e562-ff5f-48e4-8d07-c21bb3a558f7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:54.209972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716e562-ff5f-48e4-8d07-c21bb3a558f7 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-6716e562-ff5f-48e4-8d07-c21bb3a558f7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 07:15:54.220419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 07:15:54.221575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 07:15:54.222635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 652/1304] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Jul 26 07:15:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 26 07:15:54.228394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cace891-71fc-4d2a-8cc0-334258f6c5d4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:54.306393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:15:54.330313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cace891-71fc-4d2a-8cc0-334258f6c5d4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:54.366219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2fdae-7b36-44d8-8104-9eb7fd2d779f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:54.366834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2fdae-7b36-44d8-8104-9eb7fd2d779f None admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:54.367113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2fdae-7b36-44d8-8104-9eb7fd2d779f None admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:54.367388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2fdae-7b36-44d8-8104-9eb7fd2d779f None admin] PATH_INFO: `/v3/services` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:54.368065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2fdae-7b36-44d8-8104-9eb7fd2d779f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:15:54.368369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2fdae-7b36-44d8-8104-9eb7fd2d779f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:54.369774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2fdae-7b36-44d8-8104-9eb7fd2d779f None admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:54.375316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 653/1305] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Jul 26 07:15:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:15:54.381453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716e562-ff5f-48e4-8d07-c21bb3a558f7 None admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:15:54.487291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716e562-ff5f-48e4-8d07-c21bb3a558f7 None admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:15:54.522393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc2ab3-b609-42c1-81dd-0d3388d85f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:15:54.523012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc2ab3-b609-42c1-81dd-0d3388d85f88 None admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:15:54.523296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc2ab3-b609-42c1-81dd-0d3388d85f88 None admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:15:54.523571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc2ab3-b609-42c1-81dd-0d3388d85f88 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:15:54.524273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc2ab3-b609-42c1-81dd-0d3388d85f88 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:15:54.525649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cc2ab3-b609-42c1-81dd-0d3388d85f88 None admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:15:54.551863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 653/1306] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:15:54 2022] POST /identity/v3/endpoints => generated 331 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:16:04.371998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=30 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:16:04.547249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:16:58.209786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb35919-f839-4f15-a594-f26dd081800f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:16:58.210403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb35919-f839-4f15-a594-f26dd081800f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:16:58.210403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb35919-f839-4f15-a594-f26dd081800f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:16:58.212018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 654/1307] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:16:58.223414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff32c8f-8a03-48b4-8068-663fcffc9814 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:16:58.223967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff32c8f-8a03-48b4-8068-663fcffc9814 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:16:58.224060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff32c8f-8a03-48b4-8068-663fcffc9814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:16:58.367972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fff32c8f-8a03-48b4-8068-663fcffc9814 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:16:58.369809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fff32c8f-8a03-48b4-8068-663fcffc9814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:16:58.554466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 654/1308] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:16:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:16:58.605830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd3e83-100a-4759-bf53-ec56b5a8aabe None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:16:58.606406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd3e83-100a-4759-bf53-ec56b5a8aabe None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:16:58.606706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd3e83-100a-4759-bf53-ec56b5a8aabe None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:16:58.608121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 655/1309] 10.222.0.39 () {54 vars in 965 bytes} [Tue Jul 26 07:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:16:58.615668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23944e96-aed4-4b34-84a0-4607c968ad59 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:16:58.615959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23944e96-aed4-4b34-84a0-4607c968ad59 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:16:58.616235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23944e96-aed4-4b34-84a0-4607c968ad59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:16:58.696311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-23944e96-aed4-4b34-84a0-4607c968ad59 None None] MFA Rules not processed for user `1911678a39f749939dc8476617a884fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:16:58.697565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-23944e96-aed4-4b34-84a0-4607c968ad59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:16:58.725180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:16:58.799109 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 655/1310] 10.222.0.39 () {58 vars in 1062 bytes} [Tue Jul 26 07:16:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:16:58.809447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd3e83-100a-4759-bf53-ec56b5a8aabe None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:16:58.811362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39fd3e83-100a-4759-bf53-ec56b5a8aabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:16:58.947275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd3e83-100a-4759-bf53-ec56b5a8aabe None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:16:59.030559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6db6260-517e-4201-b8db-fd99911eed8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:16:59.031218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db6260-517e-4201-b8db-fd99911eed8b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:16:59.031493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db6260-517e-4201-b8db-fd99911eed8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:16:59.031748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6db6260-517e-4201-b8db-fd99911eed8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:16:59.033456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6db6260-517e-4201-b8db-fd99911eed8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:16:59.098129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:16:59.171431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db6260-517e-4201-b8db-fd99911eed8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:16:59.172892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6db6260-517e-4201-b8db-fd99911eed8b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:16:59.196627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 656/1311] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:01.535552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88797be8-9637-47f7-a9fe-40e66942423a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:01.536138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88797be8-9637-47f7-a9fe-40e66942423a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:01.536138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88797be8-9637-47f7-a9fe-40e66942423a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:01.537794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 656/1312] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:01.543537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4261b-681b-44eb-9dce-94683303e4b8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:01.544113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4261b-681b-44eb-9dce-94683303e4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:01.544113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4261b-681b-44eb-9dce-94683303e4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:01.645803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-43f4261b-681b-44eb-9dce-94683303e4b8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:01.647530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-43f4261b-681b-44eb-9dce-94683303e4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:01.794868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 657/1313] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:01.834759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88797be8-9637-47f7-a9fe-40e66942423a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:01.835703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88797be8-9637-47f7-a9fe-40e66942423a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:01.960829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88797be8-9637-47f7-a9fe-40e66942423a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:02.041331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:02.044333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1f267-b1da-4804-8e90-c7351abef449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:02.045155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f267-b1da-4804-8e90-c7351abef449 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:02.045285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f267-b1da-4804-8e90-c7351abef449 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:02.045520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f267-b1da-4804-8e90-c7351abef449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:02.047781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4ba1f267-b1da-4804-8e90-c7351abef449 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:02.171872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1f267-b1da-4804-8e90-c7351abef449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:02.173310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1f267-b1da-4804-8e90-c7351abef449 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:02.193032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 657/1314] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:04.500566 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b9e3b-22a7-4d22-a797-0db8b97abff5 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:04.501153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b9e3b-22a7-4d22-a797-0db8b97abff5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:04.501153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b9e3b-22a7-4d22-a797-0db8b97abff5 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:04.502531 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 658/1315] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:04.507833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e613d-e6be-4809-ae62-aa082add9336 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:04.508263 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e613d-e6be-4809-ae62-aa082add9336 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:04.508355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166e613d-e6be-4809-ae62-aa082add9336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:04.577983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-166e613d-e6be-4809-ae62-aa082add9336 None None] MFA Rules not processed for user `5c36c5fbc8c04f3f8235a61fc5992850`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:04.579221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-166e613d-e6be-4809-ae62-aa082add9336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:04.634530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:04.682742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 658/1316] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:04.734526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26668-8f24-4136-a249-71f6c117f2bd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:04.734808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26668-8f24-4136-a249-71f6c117f2bd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:04.735109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c26668-8f24-4136-a249-71f6c117f2bd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:04.736864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 659/1317] 10.222.0.39 () {54 vars in 965 bytes} [Tue Jul 26 07:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:04.743098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721da6a2-d366-48c9-8cb6-80c3d3acd9ba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:04.743380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721da6a2-d366-48c9-8cb6-80c3d3acd9ba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:04.743654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721da6a2-d366-48c9-8cb6-80c3d3acd9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:04.822094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-721da6a2-d366-48c9-8cb6-80c3d3acd9ba None None] MFA Rules not processed for user `1911678a39f749939dc8476617a884fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:04.823305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-721da6a2-d366-48c9-8cb6-80c3d3acd9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:04.905396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 659/1318] 10.222.0.39 () {58 vars in 1062 bytes} [Tue Jul 26 07:17:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:04.914597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c26668-8f24-4136-a249-71f6c117f2bd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:04.916152 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b7c26668-8f24-4136-a249-71f6c117f2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:04.926405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:05.026460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c26668-8f24-4136-a249-71f6c117f2bd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:05.100516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b24971-7813-406e-b842-488976a210e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:05.101139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b24971-7813-406e-b842-488976a210e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:05.101411 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b24971-7813-406e-b842-488976a210e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:05.101711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b24971-7813-406e-b842-488976a210e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:05.103348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07b24971-7813-406e-b842-488976a210e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:05.166849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:05.215382 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b24971-7813-406e-b842-488976a210e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:05.216821 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b24971-7813-406e-b842-488976a210e9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:05.235675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 660/1319] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:04 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:08.213362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c01ce-fa23-4332-a49b-fd54af338d41 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:08.213963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c01ce-fa23-4332-a49b-fd54af338d41 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:08.213963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c01ce-fa23-4332-a49b-fd54af338d41 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:08.215311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 660/1320] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:08.220985 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804d3f5-dc46-4240-9f97-345a7dbdfcd0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:08.221458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804d3f5-dc46-4240-9f97-345a7dbdfcd0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:08.221555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804d3f5-dc46-4240-9f97-345a7dbdfcd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:08.298147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2804d3f5-dc46-4240-9f97-345a7dbdfcd0 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:08.299379 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2804d3f5-dc46-4240-9f97-345a7dbdfcd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:08.399049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 661/1321] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:08.437434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c01ce-fa23-4332-a49b-fd54af338d41 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:08.439028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fc8c01ce-fa23-4332-a49b-fd54af338d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:08.493728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:08.549240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c01ce-fa23-4332-a49b-fd54af338d41 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:08.630994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1a50a-13e4-4b03-a8c7-738e8f9a6803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:08.631631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a50a-13e4-4b03-a8c7-738e8f9a6803 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:08.631907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a50a-13e4-4b03-a8c7-738e8f9a6803 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:08.632173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a50a-13e4-4b03-a8c7-738e8f9a6803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:08.633841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a1e1a50a-13e4-4b03-a8c7-738e8f9a6803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:08.740797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:08.755117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1a50a-13e4-4b03-a8c7-738e8f9a6803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:08.756621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1a50a-13e4-4b03-a8c7-738e8f9a6803 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:08.774735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 661/1322] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:11.480418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db3f4d-0dc0-4ab9-a9df-ec49784429f2 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:11.480985 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db3f4d-0dc0-4ab9-a9df-ec49784429f2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:11.480985 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db3f4d-0dc0-4ab9-a9df-ec49784429f2 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:11.482424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 662/1323] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:11.488471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ac16f-a97e-4579-b8f8-8e2ffb7cbfe0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:11.488944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ac16f-a97e-4579-b8f8-8e2ffb7cbfe0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:11.489129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ac16f-a97e-4579-b8f8-8e2ffb7cbfe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:11.558943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-519ac16f-a97e-4579-b8f8-8e2ffb7cbfe0 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:11.562003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-519ac16f-a97e-4579-b8f8-8e2ffb7cbfe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:11.660754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 662/1324] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:11.697892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db3f4d-0dc0-4ab9-a9df-ec49784429f2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:11.699639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-89db3f4d-0dc0-4ab9-a9df-ec49784429f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:11.722005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:11.816151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db3f4d-0dc0-4ab9-a9df-ec49784429f2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:11.895726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29020ea8-b961-4b3f-a3be-6506f201b4dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:11.896371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29020ea8-b961-4b3f-a3be-6506f201b4dd service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:11.896680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29020ea8-b961-4b3f-a3be-6506f201b4dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:11.896952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29020ea8-b961-4b3f-a3be-6506f201b4dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:11.898611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-29020ea8-b961-4b3f-a3be-6506f201b4dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:11.969447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:12.008799 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29020ea8-b961-4b3f-a3be-6506f201b4dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:12.010301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29020ea8-b961-4b3f-a3be-6506f201b4dd service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:12.028866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 663/1325] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:14.063104 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e0aca-65a4-4e9c-b4c1-aea965594c17 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:14.063874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e0aca-65a4-4e9c-b4c1-aea965594c17 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:14.063874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e0aca-65a4-4e9c-b4c1-aea965594c17 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:14.065047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 663/1326] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:14.071560 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedd604-076a-45d1-9193-cf5997029fa8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:14.072103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedd604-076a-45d1-9193-cf5997029fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:14.072488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedd604-076a-45d1-9193-cf5997029fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:14.195577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fcedd604-076a-45d1-9193-cf5997029fa8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:14.197455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fcedd604-076a-45d1-9193-cf5997029fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:14.378547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 664/1327] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:14.418932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e0aca-65a4-4e9c-b4c1-aea965594c17 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:14.420901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5c1e0aca-65a4-4e9c-b4c1-aea965594c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:14.495815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:14.549602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e0aca-65a4-4e9c-b4c1-aea965594c17 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:14.665356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea4418-c3f1-4ca4-8468-4d978577e722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:14.666205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea4418-c3f1-4ca4-8468-4d978577e722 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:14.666644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea4418-c3f1-4ca4-8468-4d978577e722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:14.667021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea4418-c3f1-4ca4-8468-4d978577e722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:14.668990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f3ea4418-c3f1-4ca4-8468-4d978577e722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:14.819406 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:14.822787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea4418-c3f1-4ca4-8468-4d978577e722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:14.824274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea4418-c3f1-4ca4-8468-4d978577e722 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:14.847120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 664/1328] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:17:17.225312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48031129-56b8-4f85-be02-b918c5962005 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:17.226118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48031129-56b8-4f85-be02-b918c5962005 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:17.226118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48031129-56b8-4f85-be02-b918c5962005 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:17.227589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 665/1329] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:17.234691 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea4e7f-05c9-41b9-a41c-396f75128484 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:17.235647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea4e7f-05c9-41b9-a41c-396f75128484 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:17.235647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ea4e7f-05c9-41b9-a41c-396f75128484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:17.322075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-51ea4e7f-05c9-41b9-a41c-396f75128484 None None] MFA Rules not processed for user `5c36c5fbc8c04f3f8235a61fc5992850`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:17.323888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-51ea4e7f-05c9-41b9-a41c-396f75128484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:17.472578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 665/1330] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:17 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:17.515187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48031129-56b8-4f85-be02-b918c5962005 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:17.516958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-48031129-56b8-4f85-be02-b918c5962005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:17.564440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:17.666812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48031129-56b8-4f85-be02-b918c5962005 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:17.772191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ebabc-c90b-409b-87a0-0a2edf7835e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:17.772735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ebabc-c90b-409b-87a0-0a2edf7835e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:17.773018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ebabc-c90b-409b-87a0-0a2edf7835e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:17.773287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ebabc-c90b-409b-87a0-0a2edf7835e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:17.775117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a41ebabc-c90b-409b-87a0-0a2edf7835e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:17.871519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:17.936688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ebabc-c90b-409b-87a0-0a2edf7835e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:17.938428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ebabc-c90b-409b-87a0-0a2edf7835e2 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:17.964733 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 666/1331] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:17 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:20.534396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173d559-53e6-4917-b594-b6bc4617330c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:20.535287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173d559-53e6-4917-b594-b6bc4617330c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:20.535287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173d559-53e6-4917-b594-b6bc4617330c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:20.536506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 666/1332] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:20.550909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2f612-9e43-4922-8e47-599399881c43 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:20.551660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2f612-9e43-4922-8e47-599399881c43 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:20.551660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2f612-9e43-4922-8e47-599399881c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:20.638106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2ff2f612-9e43-4922-8e47-599399881c43 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:20.639351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2ff2f612-9e43-4922-8e47-599399881c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:20.770913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 667/1333] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:20.814323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173d559-53e6-4917-b594-b6bc4617330c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:20.816059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9173d559-53e6-4917-b594-b6bc4617330c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:20.909758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:20.956202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173d559-53e6-4917-b594-b6bc4617330c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:21.058323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1f11b-76de-45b4-9e90-23d889b246c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:21.058897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1f11b-76de-45b4-9e90-23d889b246c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:21.059301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1f11b-76de-45b4-9e90-23d889b246c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:21.059577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1f11b-76de-45b4-9e90-23d889b246c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:21.061455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-95b1f11b-76de-45b4-9e90-23d889b246c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:21.201465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1f11b-76de-45b4-9e90-23d889b246c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:21.203140 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1f11b-76de-45b4-9e90-23d889b246c5 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:21.215731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:21.225414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 667/1334] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:23.359215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378341c-5856-4bb1-a4ce-ef425fb92e88 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:23.359849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378341c-5856-4bb1-a4ce-ef425fb92e88 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:23.359849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378341c-5856-4bb1-a4ce-ef425fb92e88 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:23.361209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 668/1335] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:23.367987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827e359-c4af-42f7-8376-5a8f35680fc8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:23.368757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827e359-c4af-42f7-8376-5a8f35680fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:23.368757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827e359-c4af-42f7-8376-5a8f35680fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:23.470952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7827e359-c4af-42f7-8376-5a8f35680fc8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:23.472587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7827e359-c4af-42f7-8376-5a8f35680fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:23.606228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 668/1336] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:17:23.660761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378341c-5856-4bb1-a4ce-ef425fb92e88 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:23.662383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0378341c-5856-4bb1-a4ce-ef425fb92e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:23.707557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:23.801271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378341c-5856-4bb1-a4ce-ef425fb92e88 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:23.890004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb4df2-93bc-44fd-b773-b3548bc30106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:23.890699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb4df2-93bc-44fd-b773-b3548bc30106 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:23.890995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb4df2-93bc-44fd-b773-b3548bc30106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:23.891270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb4df2-93bc-44fd-b773-b3548bc30106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:23.893150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-83fb4df2-93bc-44fd-b773-b3548bc30106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:23.985346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:24.016856 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb4df2-93bc-44fd-b773-b3548bc30106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:24.018405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb4df2-93bc-44fd-b773-b3548bc30106 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:24.041191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 669/1337] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:26.970890 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185b1d6-a6f7-4ce8-bf41-b762ed1ca716 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:26.971631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185b1d6-a6f7-4ce8-bf41-b762ed1ca716 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:26.971631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185b1d6-a6f7-4ce8-bf41-b762ed1ca716 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:26.973423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 669/1338] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:26.979210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14dac0-2a14-4722-abdf-f05679703163 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:26.979725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14dac0-2a14-4722-abdf-f05679703163 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:26.979845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14dac0-2a14-4722-abdf-f05679703163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:27.062756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-da14dac0-2a14-4722-abdf-f05679703163 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:27.064071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-da14dac0-2a14-4722-abdf-f05679703163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:27.175575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 670/1339] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:27.213538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185b1d6-a6f7-4ce8-bf41-b762ed1ca716 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:27.215120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d185b1d6-a6f7-4ce8-bf41-b762ed1ca716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:27.303291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:27.336102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185b1d6-a6f7-4ce8-bf41-b762ed1ca716 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:27.415316 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c095bdb-804c-4cc4-b38f-d259f1c6c946 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:27.415950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c095bdb-804c-4cc4-b38f-d259f1c6c946 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:27.416224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c095bdb-804c-4cc4-b38f-d259f1c6c946 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:27.416489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c095bdb-804c-4cc4-b38f-d259f1c6c946 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:27.418166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6c095bdb-804c-4cc4-b38f-d259f1c6c946 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:27.548153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c095bdb-804c-4cc4-b38f-d259f1c6c946 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:27.549597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c095bdb-804c-4cc4-b38f-d259f1c6c946 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:27.574020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 670/1340] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:27 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:31.941974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce418e4-de69-4ec3-8737-a25a91506f58 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:31.942526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce418e4-de69-4ec3-8737-a25a91506f58 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:31.942709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce418e4-de69-4ec3-8737-a25a91506f58 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:31.944906 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 671/1341] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:31.951809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3c789-575f-4b20-8af4-cc9e93db3734 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:31.952324 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3c789-575f-4b20-8af4-cc9e93db3734 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:31.952635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3c789-575f-4b20-8af4-cc9e93db3734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:31.959622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:32.053695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-04e3c789-575f-4b20-8af4-cc9e93db3734 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:32.057181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-04e3c789-575f-4b20-8af4-cc9e93db3734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:32.204246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 671/1342] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:17:32.244866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce418e4-de69-4ec3-8737-a25a91506f58 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:32.246500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9ce418e4-de69-4ec3-8737-a25a91506f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:32.313399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:32.379969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce418e4-de69-4ec3-8737-a25a91506f58 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:32.474431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5492c97-275e-405b-a9f2-bd290d072dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:32.475305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5492c97-275e-405b-a9f2-bd290d072dfb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:32.475791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5492c97-275e-405b-a9f2-bd290d072dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:32.476210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5492c97-275e-405b-a9f2-bd290d072dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:32.478583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5492c97-275e-405b-a9f2-bd290d072dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:32.592360 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:32.614458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5492c97-275e-405b-a9f2-bd290d072dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:32.616126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5492c97-275e-405b-a9f2-bd290d072dfb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:32.637958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 672/1343] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:35.506449 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffcf70-4a05-4762-b853-34206eaa4463 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:35.507033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffcf70-4a05-4762-b853-34206eaa4463 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:35.507033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeffcf70-4a05-4762-b853-34206eaa4463 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:35.508482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 672/1344] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:35.515284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c4b5f-4a9c-4158-b82e-80690435e90d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:35.515932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c4b5f-4a9c-4158-b82e-80690435e90d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:35.516078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835c4b5f-4a9c-4158-b82e-80690435e90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:35.624539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-835c4b5f-4a9c-4158-b82e-80690435e90d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:35.627084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-835c4b5f-4a9c-4158-b82e-80690435e90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:35.782486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 673/1345] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:17:35.827133 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffcf70-4a05-4762-b853-34206eaa4463 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:35.829955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aeffcf70-4a05-4762-b853-34206eaa4463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:35.970350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:35.999896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeffcf70-4a05-4762-b853-34206eaa4463 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:36.100151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3142986-0917-4ec8-8bbd-f36fccb02091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:36.100718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3142986-0917-4ec8-8bbd-f36fccb02091 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:36.101073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3142986-0917-4ec8-8bbd-f36fccb02091 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:36.102205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3142986-0917-4ec8-8bbd-f36fccb02091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:36.103064 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b3142986-0917-4ec8-8bbd-f36fccb02091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:36.241116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3142986-0917-4ec8-8bbd-f36fccb02091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:36.242806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3142986-0917-4ec8-8bbd-f36fccb02091 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:36.263531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 673/1346] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:40.748983 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d5180-8770-4bdc-a12e-655b0c60a012 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:40.749553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d5180-8770-4bdc-a12e-655b0c60a012 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:40.749714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d5180-8770-4bdc-a12e-655b0c60a012 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:40.751776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 674/1347] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:40.758888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da0207-812a-4e65-9012-4a683fe63ff7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:40.759537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da0207-812a-4e65-9012-4a683fe63ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:40.759752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da0207-812a-4e65-9012-4a683fe63ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:40.780028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:40.866377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-15da0207-812a-4e65-9012-4a683fe63ff7 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:40.868343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-15da0207-812a-4e65-9012-4a683fe63ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:41.031330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 674/1348] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:41.073959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d5180-8770-4bdc-a12e-655b0c60a012 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:41.076677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-568d5180-8770-4bdc-a12e-655b0c60a012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:41.152056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:41.213675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d5180-8770-4bdc-a12e-655b0c60a012 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:41.303944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b69e80-646e-468e-a1a4-8c5d0a641551 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:41.304589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b69e80-646e-468e-a1a4-8c5d0a641551 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:41.304876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b69e80-646e-468e-a1a4-8c5d0a641551 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:41.305137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b69e80-646e-468e-a1a4-8c5d0a641551 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:41.307274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-59b69e80-646e-468e-a1a4-8c5d0a641551 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:41.452109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:41.457564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b69e80-646e-468e-a1a4-8c5d0a641551 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:41.459285 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b69e80-646e-468e-a1a4-8c5d0a641551 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:41.481356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 675/1349] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:46.542757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041ddec-21fa-4c90-a198-b7ce156014e2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:46.543605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041ddec-21fa-4c90-a198-b7ce156014e2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:46.543885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041ddec-21fa-4c90-a198-b7ce156014e2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:46.545262 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 675/1350] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:46.559602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a9ac0-2077-4e92-bb0f-61f979968415 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:46.560055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a9ac0-2077-4e92-bb0f-61f979968415 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:46.560150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a9ac0-2077-4e92-bb0f-61f979968415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:46.705125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e49a9ac0-2077-4e92-bb0f-61f979968415 None None] MFA Rules not processed for user `51a5093a60ef45919ab6e159adf58336`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:46.706358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e49a9ac0-2077-4e92-bb0f-61f979968415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:46.840404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 676/1351] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:17:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:50.984689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58800d8f-7231-4f20-84cc-36b32a609e5d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:50.985614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58800d8f-7231-4f20-84cc-36b32a609e5d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:50.986096 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58800d8f-7231-4f20-84cc-36b32a609e5d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:50.987815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 676/1352] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:50.993263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c316a94-8397-4fb4-8b60-0c2f7e1851b1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:50.993762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c316a94-8397-4fb4-8b60-0c2f7e1851b1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:50.994121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c316a94-8397-4fb4-8b60-0c2f7e1851b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:51.027601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=59 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:51.055536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:51.079540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7c316a94-8397-4fb4-8b60-0c2f7e1851b1 None None] MFA Rules not processed for user `51a5093a60ef45919ab6e159adf58336`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:51.080815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c316a94-8397-4fb4-8b60-0c2f7e1851b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:51.194709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 677/1353] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:17:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:52.673927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822c77-a25b-4e67-ac5f-3d83ed1859e7 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:52.674477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822c77-a25b-4e67-ac5f-3d83ed1859e7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:52.674477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27822c77-a25b-4e67-ac5f-3d83ed1859e7 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:52.676109 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 677/1354] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:17:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:52.681890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b59e0-3b84-4777-9d0d-0b322fab1679 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:52.682452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b59e0-3b84-4777-9d0d-0b322fab1679 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:52.682452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b59e0-3b84-4777-9d0d-0b322fab1679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:52.767896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-370b59e0-3b84-4777-9d0d-0b322fab1679 None None] MFA Rules not processed for user `51a5093a60ef45919ab6e159adf58336`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:52.769132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-370b59e0-3b84-4777-9d0d-0b322fab1679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:52.814699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:52.883841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 678/1355] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:17:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:56.216479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38f2f-e477-446b-b86a-6685179b1319 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:56.217011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38f2f-e477-446b-b86a-6685179b1319 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:56.217011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd38f2f-e477-446b-b86a-6685179b1319 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:56.218399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 678/1356] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:56.227178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885fb08-a60b-43d8-90d8-4b6532690c41 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:56.227864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885fb08-a60b-43d8-90d8-4b6532690c41 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:56.227864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885fb08-a60b-43d8-90d8-4b6532690c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:56.322788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f885fb08-a60b-43d8-90d8-4b6532690c41 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:56.322788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f885fb08-a60b-43d8-90d8-4b6532690c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:56.441922 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 679/1357] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:56.487675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9611e-77a1-4b52-bc67-79d972066954 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:56.488316 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9611e-77a1-4b52-bc67-79d972066954 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:56.488741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc9611e-77a1-4b52-bc67-79d972066954 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:56.490399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 679/1358] 10.222.0.39 () {54 vars in 962 bytes} [Tue Jul 26 07:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:56.511183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a1b76-4ba0-4553-9250-229943488869 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:56.511492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a1b76-4ba0-4553-9250-229943488869 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:56.511751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a1b76-4ba0-4553-9250-229943488869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:56.530443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:56.603422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4d2a1b76-4ba0-4553-9250-229943488869 None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:56.604762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4d2a1b76-4ba0-4553-9250-229943488869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:56.730978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 680/1359] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Jul 26 07:17:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:17:56.742305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9611e-77a1-4b52-bc67-79d972066954 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:56.744065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dfc9611e-77a1-4b52-bc67-79d972066954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:56.897812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc9611e-77a1-4b52-bc67-79d972066954 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:56.981227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:57.032967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b43e65-6300-4ef0-8f3e-aeab65be772b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:57.034276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b43e65-6300-4ef0-8f3e-aeab65be772b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:57.034630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b43e65-6300-4ef0-8f3e-aeab65be772b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:57.035013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b43e65-6300-4ef0-8f3e-aeab65be772b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:57.037039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-60b43e65-6300-4ef0-8f3e-aeab65be772b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:57.156894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b43e65-6300-4ef0-8f3e-aeab65be772b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:17:57.158762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b43e65-6300-4ef0-8f3e-aeab65be772b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:17:57.180770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 680/1360] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:17:59.427598 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0102376-e733-46cd-890f-0b92129d91bb None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:59.427598 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0102376-e733-46cd-890f-0b92129d91bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:59.427598 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0102376-e733-46cd-890f-0b92129d91bb None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:59.430179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 681/1361] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:17:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:17:59.432773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeab141-8128-40fb-86c9-1ca3699b40e4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:59.433219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeab141-8128-40fb-86c9-1ca3699b40e4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:59.433396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeab141-8128-40fb-86c9-1ca3699b40e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:59.505545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6aeab141-8128-40fb-86c9-1ca3699b40e4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:17:59.506886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6aeab141-8128-40fb-86c9-1ca3699b40e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:59.549482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:59.614788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 681/1362] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:17:59.659819 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0102376-e733-46cd-890f-0b92129d91bb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:17:59.661341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0102376-e733-46cd-890f-0b92129d91bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:17:59.747804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:17:59.844134 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0102376-e733-46cd-890f-0b92129d91bb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:17:59.984273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3e8c3-b66f-46ea-bbb2-b08037007916 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:17:59.984975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3e8c3-b66f-46ea-bbb2-b08037007916 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:17:59.985366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3e8c3-b66f-46ea-bbb2-b08037007916 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:17:59.985560 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3e8c3-b66f-46ea-bbb2-b08037007916 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:17:59.987909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-faa3e8c3-b66f-46ea-bbb2-b08037007916 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:00.016300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:00.099285 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3e8c3-b66f-46ea-bbb2-b08037007916 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:00.100974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3e8c3-b66f-46ea-bbb2-b08037007916 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:00.126108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 682/1363] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:17:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:02.668336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec124fe-d9fc-4951-9035-79bdf39d2d6a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:02.669194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec124fe-d9fc-4951-9035-79bdf39d2d6a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:02.669194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec124fe-d9fc-4951-9035-79bdf39d2d6a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:02.671906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 682/1364] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:02.680032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462470cc-6844-4e7c-a9bd-7d0bfc3001de None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:02.680807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462470cc-6844-4e7c-a9bd-7d0bfc3001de None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:02.680807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462470cc-6844-4e7c-a9bd-7d0bfc3001de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:02.780747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-462470cc-6844-4e7c-a9bd-7d0bfc3001de None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:02.782730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-462470cc-6844-4e7c-a9bd-7d0bfc3001de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:02.946677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:02.951864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 683/1365] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:18:03.029577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932434-9f54-4163-846f-ae3d032fc535 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:03.030240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932434-9f54-4163-846f-ae3d032fc535 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:03.030714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb932434-9f54-4163-846f-ae3d032fc535 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:03.032355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 683/1366] 10.222.0.39 () {54 vars in 962 bytes} [Tue Jul 26 07:18:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:03.042062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40690d42-74be-47a1-a1e9-023c3dcdfb0a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:03.042545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40690d42-74be-47a1-a1e9-023c3dcdfb0a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:03.042988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40690d42-74be-47a1-a1e9-023c3dcdfb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:03.194196 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-40690d42-74be-47a1-a1e9-023c3dcdfb0a None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:03.196203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40690d42-74be-47a1-a1e9-023c3dcdfb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:03.366924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 684/1367] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Jul 26 07:18:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:03.375270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb932434-9f54-4163-846f-ae3d032fc535 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:03.377032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb932434-9f54-4163-846f-ae3d032fc535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:03.568671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb932434-9f54-4163-846f-ae3d032fc535 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:03.594777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:03.710247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ebbab-bac7-4f9c-bda1-42da7e051294 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:03.711231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ebbab-bac7-4f9c-bda1-42da7e051294 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:03.711231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ebbab-bac7-4f9c-bda1-42da7e051294 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:03.711537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ebbab-bac7-4f9c-bda1-42da7e051294 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:03.713038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-316ebbab-bac7-4f9c-bda1-42da7e051294 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:03.813102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ebbab-bac7-4f9c-bda1-42da7e051294 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:03.814969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ebbab-bac7-4f9c-bda1-42da7e051294 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:03.835172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 684/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:04.879550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5eca-2359-4cdd-8273-4e61727860e6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:04.880168 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5eca-2359-4cdd-8273-4e61727860e6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:04.880739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c5eca-2359-4cdd-8273-4e61727860e6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:04.884019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 685/1369] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:04.890674 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb31c-a43c-406c-a33a-6a31bbd70afc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:04.891524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb31c-a43c-406c-a33a-6a31bbd70afc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:04.891813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdb31c-a43c-406c-a33a-6a31bbd70afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:04.932165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:04.996987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6ecdb31c-a43c-406c-a33a-6a31bbd70afc None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:04.999034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6ecdb31c-a43c-406c-a33a-6a31bbd70afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:05.177622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 685/1370] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:18:05.536399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b279902-3e05-4981-ba47-7a76a8740e16 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:05.536783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b279902-3e05-4981-ba47-7a76a8740e16 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:05.537069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b279902-3e05-4981-ba47-7a76a8740e16 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:05.538881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 686/1371] 10.222.0.39 () {54 vars in 964 bytes} [Tue Jul 26 07:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:05.544342 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f630e0d-e27a-4027-94de-4838390d0307 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:05.544648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f630e0d-e27a-4027-94de-4838390d0307 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:05.544919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f630e0d-e27a-4027-94de-4838390d0307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:05.650394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3f630e0d-e27a-4027-94de-4838390d0307 None None] MFA Rules not processed for user `becff3f1dacb43c8a014c95456e86bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:05.652366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3f630e0d-e27a-4027-94de-4838390d0307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:05.745507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:05.808687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 686/1372] 10.222.0.39 () {58 vars in 1061 bytes} [Tue Jul 26 07:18:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:18:05.817900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b279902-3e05-4981-ba47-7a76a8740e16 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:05.820081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b279902-3e05-4981-ba47-7a76a8740e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:05.881992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:05.948411 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b279902-3e05-4981-ba47-7a76a8740e16 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:06.066887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46392e2c-7bb2-42ac-8bb8-ddaa327cdcfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:06.067924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46392e2c-7bb2-42ac-8bb8-ddaa327cdcfc service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:06.068479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46392e2c-7bb2-42ac-8bb8-ddaa327cdcfc service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:06.068925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46392e2c-7bb2-42ac-8bb8-ddaa327cdcfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:06.072013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-46392e2c-7bb2-42ac-8bb8-ddaa327cdcfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:06.298261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:06.321681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46392e2c-7bb2-42ac-8bb8-ddaa327cdcfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:06.323392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46392e2c-7bb2-42ac-8bb8-ddaa327cdcfc service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:06.348586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 687/1373] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:06.422020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893862d-020f-4612-af95-a2d9b583c5df None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:06.422569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893862d-020f-4612-af95-a2d9b583c5df None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:06.422569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893862d-020f-4612-af95-a2d9b583c5df None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:06.423978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 687/1374] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:18:06.433527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bada992-7f70-4b93-8e51-0d8c3f206142 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:06.434072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bada992-7f70-4b93-8e51-0d8c3f206142 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:06.434573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bada992-7f70-4b93-8e51-0d8c3f206142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:06.538650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0bada992-7f70-4b93-8e51-0d8c3f206142 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:06.540652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0bada992-7f70-4b93-8e51-0d8c3f206142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:06.667177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 688/1375] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:06.733994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893862d-020f-4612-af95-a2d9b583c5df None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:06.736163 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d893862d-020f-4612-af95-a2d9b583c5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:06.910869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893862d-020f-4612-af95-a2d9b583c5df None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:06.978236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:07.078496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15faddb-bae1-468d-86b3-305f551df925 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:07.079183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15faddb-bae1-468d-86b3-305f551df925 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:07.079527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15faddb-bae1-468d-86b3-305f551df925 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:07.079849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15faddb-bae1-468d-86b3-305f551df925 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:07.081664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f15faddb-bae1-468d-86b3-305f551df925 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:07.211673 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15faddb-bae1-468d-86b3-305f551df925 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:07.214975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15faddb-bae1-468d-86b3-305f551df925 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:07.243857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 688/1376] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:09.461370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7905dcd-cd9c-43c7-85a4-d95f2ae1821b None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:09.462105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7905dcd-cd9c-43c7-85a4-d95f2ae1821b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:09.462105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7905dcd-cd9c-43c7-85a4-d95f2ae1821b None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:09.463571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 689/1377] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:09.469727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234020b7-e94c-44d6-92b0-6fb9e83f61d5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:09.470317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234020b7-e94c-44d6-92b0-6fb9e83f61d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:09.470537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234020b7-e94c-44d6-92b0-6fb9e83f61d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:09.568409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-234020b7-e94c-44d6-92b0-6fb9e83f61d5 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:09.569940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-234020b7-e94c-44d6-92b0-6fb9e83f61d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:09.579993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:09.708807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 689/1378] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:09.775147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7905dcd-cd9c-43c7-85a4-d95f2ae1821b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:09.777212 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7905dcd-cd9c-43c7-85a4-d95f2ae1821b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:09.868378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:10.000430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7905dcd-cd9c-43c7-85a4-d95f2ae1821b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:10.148756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0341e2-017c-4c2e-acd5-2e7840292979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:10.149613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0341e2-017c-4c2e-acd5-2e7840292979 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:10.150022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0341e2-017c-4c2e-acd5-2e7840292979 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:10.150381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0341e2-017c-4c2e-acd5-2e7840292979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:10.152264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf0341e2-017c-4c2e-acd5-2e7840292979 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:10.175525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:10.294935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0341e2-017c-4c2e-acd5-2e7840292979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:10.299761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0341e2-017c-4c2e-acd5-2e7840292979 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:10.340185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 690/1379] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:18:12.648550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f7fdf-ae72-43d0-bdc4-3d0095dd7e5b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:12.649169 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f7fdf-ae72-43d0-bdc4-3d0095dd7e5b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:12.649169 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f7fdf-ae72-43d0-bdc4-3d0095dd7e5b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:12.650645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 690/1380] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:18:12.656485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73431626-6ef3-43fd-9e66-0290c6fa2e03 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:12.657342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73431626-6ef3-43fd-9e66-0290c6fa2e03 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:12.657342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73431626-6ef3-43fd-9e66-0290c6fa2e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:12.760665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-73431626-6ef3-43fd-9e66-0290c6fa2e03 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:12.762631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-73431626-6ef3-43fd-9e66-0290c6fa2e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:12.920859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:12.934000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 691/1381] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:18:12.973879 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f7fdf-ae72-43d0-bdc4-3d0095dd7e5b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:12.975474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b73f7fdf-ae72-43d0-bdc4-3d0095dd7e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:13.169299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:13.186953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f7fdf-ae72-43d0-bdc4-3d0095dd7e5b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:13.384227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3373e411-fddf-4dd5-b1bf-d6aae585e989 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:13.385474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373e411-fddf-4dd5-b1bf-d6aae585e989 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:13.385966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373e411-fddf-4dd5-b1bf-d6aae585e989 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:13.386431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3373e411-fddf-4dd5-b1bf-d6aae585e989 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:13.389364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3373e411-fddf-4dd5-b1bf-d6aae585e989 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:13.560402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373e411-fddf-4dd5-b1bf-d6aae585e989 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:13.562092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3373e411-fddf-4dd5-b1bf-d6aae585e989 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:13.582577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:13.593505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 691/1382] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 620 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:16.121525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c950f6d-4f7d-4b31-92b6-34a2e436aa63 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:16.122166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c950f6d-4f7d-4b31-92b6-34a2e436aa63 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:16.122264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c950f6d-4f7d-4b31-92b6-34a2e436aa63 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:16.124159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 692/1383] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:16.131311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbbbd8-6c42-4837-b754-a512fcbbcd33 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:16.131880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbbbd8-6c42-4837-b754-a512fcbbcd33 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:16.131880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cbbbd8-6c42-4837-b754-a512fcbbcd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:16.207378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-97cbbbd8-6c42-4837-b754-a512fcbbcd33 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:16.208602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-97cbbbd8-6c42-4837-b754-a512fcbbcd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:16.331277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 692/1384] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:16.387759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c950f6d-4f7d-4b31-92b6-34a2e436aa63 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:16.389439 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c950f6d-4f7d-4b31-92b6-34a2e436aa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:16.560731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c950f6d-4f7d-4b31-92b6-34a2e436aa63 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:16.649158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:16.713891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fc2f0-1880-4211-a98b-8ccb56fc48f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:16.715288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fc2f0-1880-4211-a98b-8ccb56fc48f9 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:16.715681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fc2f0-1880-4211-a98b-8ccb56fc48f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:16.716157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fc2f0-1880-4211-a98b-8ccb56fc48f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:16.719025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f53fc2f0-1880-4211-a98b-8ccb56fc48f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:16.821700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fc2f0-1880-4211-a98b-8ccb56fc48f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:16.823504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53fc2f0-1880-4211-a98b-8ccb56fc48f9 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:16.843253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 693/1385] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:19.328550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863eed5-93ec-4986-83b0-e431c6c40cd4 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:19.329230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863eed5-93ec-4986-83b0-e431c6c40cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:19.329230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863eed5-93ec-4986-83b0-e431c6c40cd4 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:19.330740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 693/1386] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:19.338484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b2a47-ee11-4245-a911-09b87c5e3e07 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:19.339150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b2a47-ee11-4245-a911-09b87c5e3e07 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:19.339336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b2a47-ee11-4245-a911-09b87c5e3e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:19.430999 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-282b2a47-ee11-4245-a911-09b87c5e3e07 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:19.432643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-282b2a47-ee11-4245-a911-09b87c5e3e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:19.477333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:19.572493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 694/1387] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:18:19.625057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863eed5-93ec-4986-83b0-e431c6c40cd4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:19.626687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2863eed5-93ec-4986-83b0-e431c6c40cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:19.727878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:19.809330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863eed5-93ec-4986-83b0-e431c6c40cd4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:19.941568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd08cfa-475c-4aeb-aa24-7aab535116fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:19.942207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd08cfa-475c-4aeb-aa24-7aab535116fb service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:19.942505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd08cfa-475c-4aeb-aa24-7aab535116fb service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:19.942808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd08cfa-475c-4aeb-aa24-7aab535116fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:19.944564 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fbd08cfa-475c-4aeb-aa24-7aab535116fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:19.999577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:20.094844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd08cfa-475c-4aeb-aa24-7aab535116fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:20.095785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd08cfa-475c-4aeb-aa24-7aab535116fb service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:20.121686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 694/1388] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:22.229067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210d4ad-78cd-489b-bda3-a7c07fb4f637 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:22.229686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210d4ad-78cd-489b-bda3-a7c07fb4f637 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:22.229686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a210d4ad-78cd-489b-bda3-a7c07fb4f637 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:22.231102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 695/1389] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:22.236990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e959230-908b-4596-bb2e-29c5c494c284 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:22.237461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e959230-908b-4596-bb2e-29c5c494c284 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:22.237559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e959230-908b-4596-bb2e-29c5c494c284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:22.308574 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8e959230-908b-4596-bb2e-29c5c494c284 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:22.309872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8e959230-908b-4596-bb2e-29c5c494c284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:22.406305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb6445-025f-4794-a4fb-66be6e405a36 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:22.406904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb6445-025f-4794-a4fb-66be6e405a36 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:22.407436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb6445-025f-4794-a4fb-66be6e405a36 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:22.408839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 696/1390] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:22.414593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f95b4b-a48d-43ba-8bb9-1f68050c44d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:22.414875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f95b4b-a48d-43ba-8bb9-1f68050c44d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:22.415365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f95b4b-a48d-43ba-8bb9-1f68050c44d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:22.425029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 695/1391] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:22.470495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e959230-908b-4596-bb2e-29c5c494c284 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:22.471702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8e959230-908b-4596-bb2e-29c5c494c284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:22.486955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:22.495436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-98f95b4b-a48d-43ba-8bb9-1f68050c44d1 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:22.496811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98f95b4b-a48d-43ba-8bb9-1f68050c44d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:22.589029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e959230-908b-4596-bb2e-29c5c494c284 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:22.597075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 697/1392] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:22.653813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66aaea-366c-4a8e-a2af-8d09f97add04 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:22.654107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66aaea-366c-4a8e-a2af-8d09f97add04 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:22.654383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66aaea-366c-4a8e-a2af-8d09f97add04 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:22.655763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 698/1393] 10.222.0.39 () {54 vars in 966 bytes} [Tue Jul 26 07:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:18:22.661108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3121a2-f1d1-405c-aa52-7bac8d1fc59e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:22.661376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3121a2-f1d1-405c-aa52-7bac8d1fc59e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:22.662124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3121a2-f1d1-405c-aa52-7bac8d1fc59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:22.678600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:22.705907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cde434-2b06-48a7-9201-b75cb981e2fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:22.706548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cde434-2b06-48a7-9201-b75cb981e2fa service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:22.706828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cde434-2b06-48a7-9201-b75cb981e2fa service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:22.707102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cde434-2b06-48a7-9201-b75cb981e2fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:22.708828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-94cde434-2b06-48a7-9201-b75cb981e2fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:22.718328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:22.743466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ba3121a2-f1d1-405c-aa52-7bac8d1fc59e None None] MFA Rules not processed for user `a3c6943686c843e588df14f7c6dffb8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:22.744737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ba3121a2-f1d1-405c-aa52-7bac8d1fc59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:22.804097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cde434-2b06-48a7-9201-b75cb981e2fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:22.805787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cde434-2b06-48a7-9201-b75cb981e2fa service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:22.823932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 699/1394] 10.222.0.39 () {58 vars in 1063 bytes} [Tue Jul 26 07:18:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:18:22.825585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 696/1395] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:18:22.831711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3121a2-f1d1-405c-aa52-7bac8d1fc59e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:22.840114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3121a2-f1d1-405c-aa52-7bac8d1fc59e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:22.924148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01893776-c6e5-4f2f-b147-54788a157e4f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c6943686c843e588df14f7c6dffb8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:22.924684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01893776-c6e5-4f2f-b147-54788a157e4f service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:22.924959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01893776-c6e5-4f2f-b147-54788a157e4f service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:22.925227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01893776-c6e5-4f2f-b147-54788a157e4f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:22.926954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-01893776-c6e5-4f2f-b147-54788a157e4f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:22.961866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:23.059385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01893776-c6e5-4f2f-b147-54788a157e4f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:23.061086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01893776-c6e5-4f2f-b147-54788a157e4f service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:23.084281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 700/1396] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:23.851336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2760c-d339-475f-8597-bdf9311ff0b8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:23.851874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2760c-d339-475f-8597-bdf9311ff0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:23.852011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2760c-d339-475f-8597-bdf9311ff0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:23.926439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4ac2760c-d339-475f-8597-bdf9311ff0b8 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:23.927844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4ac2760c-d339-475f-8597-bdf9311ff0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:24.017828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:24.024108 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 697/1397] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:18:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:24.074717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47ed69-6401-41b8-8160-23757d0f4270 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:24.075311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47ed69-6401-41b8-8160-23757d0f4270 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:24.075311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47ed69-6401-41b8-8160-23757d0f4270 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:24.076813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 701/1398] 10.222.0.39 () {54 vars in 943 bytes} [Tue Jul 26 07:18:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:24.084803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2783b-e0f6-4fe2-83d7-289bfa0c1543 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:24.085096 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2783b-e0f6-4fe2-83d7-289bfa0c1543 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:24.085478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f2783b-e0f6-4fe2-83d7-289bfa0c1543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:24.187510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f6f2783b-e0f6-4fe2-83d7-289bfa0c1543 None None] MFA Rules not processed for user `64257d34b2df4af1a2016d56be8d44a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:24.188751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f6f2783b-e0f6-4fe2-83d7-289bfa0c1543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:24.310410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 698/1399] 10.222.0.39 () {58 vars in 1040 bytes} [Tue Jul 26 07:18:24 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:24.320148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47ed69-6401-41b8-8160-23757d0f4270 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:24.321762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be47ed69-6401-41b8-8160-23757d0f4270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:24.483615 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:24.487005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47ed69-6401-41b8-8160-23757d0f4270 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:24.605438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a235c-b6ef-4df5-b2d2-13bb41eff6c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:24.606113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a235c-b6ef-4df5-b2d2-13bb41eff6c8 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:24.606395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a235c-b6ef-4df5-b2d2-13bb41eff6c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:24.606649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a235c-b6ef-4df5-b2d2-13bb41eff6c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:24.608342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f15a235c-b6ef-4df5-b2d2-13bb41eff6c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:24.698771 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a235c-b6ef-4df5-b2d2-13bb41eff6c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:24.700275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a235c-b6ef-4df5-b2d2-13bb41eff6c8 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:24.709432 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 702/1400] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:18:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:18:24.900771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5a4c6-f408-4960-a1b5-09935941b7f6 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:24.901340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5a4c6-f408-4960-a1b5-09935941b7f6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:24.901340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc5a4c6-f408-4960-a1b5-09935941b7f6 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:24.902871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 699/1401] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:24.910708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae07f-fe62-40d9-8f13-f86a51261eab None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:24.911355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae07f-fe62-40d9-8f13-f86a51261eab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:24.911748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fae07f-fe62-40d9-8f13-f86a51261eab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:24.967587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:25.005705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-43fae07f-fe62-40d9-8f13-f86a51261eab None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:25.007159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-43fae07f-fe62-40d9-8f13-f86a51261eab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:25.136574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 703/1402] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:25.183678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5a4c6-f408-4960-a1b5-09935941b7f6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:25.185287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9fc5a4c6-f408-4960-a1b5-09935941b7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:25.283741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:25.434437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc5a4c6-f408-4960-a1b5-09935941b7f6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:25.606125 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eacfec9-b009-4af1-87fa-1236895a795c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:25.606773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacfec9-b009-4af1-87fa-1236895a795c service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:25.607055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacfec9-b009-4af1-87fa-1236895a795c service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:25.608957 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eacfec9-b009-4af1-87fa-1236895a795c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:25.609128 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7eacfec9-b009-4af1-87fa-1236895a795c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:25.630099 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:25.749400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacfec9-b009-4af1-87fa-1236895a795c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:25.751120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eacfec9-b009-4af1-87fa-1236895a795c service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:25.772951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 700/1403] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:27.869779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fc486-c5c8-4b6c-a461-e4eaf3ef7937 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:27.869779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fc486-c5c8-4b6c-a461-e4eaf3ef7937 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:27.869779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fc486-c5c8-4b6c-a461-e4eaf3ef7937 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:27.871053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 704/1404] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:27.876518 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554b4c3-9074-42a0-9efb-3180a8504d32 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:27.877021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554b4c3-9074-42a0-9efb-3180a8504d32 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:27.877119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554b4c3-9074-42a0-9efb-3180a8504d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:27.949234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3554b4c3-9074-42a0-9efb-3180a8504d32 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:27.950483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3554b4c3-9074-42a0-9efb-3180a8504d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:28.064832 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:28.073224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 701/1405] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:28.123750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fc486-c5c8-4b6c-a461-e4eaf3ef7937 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:28.126442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-689fc486-c5c8-4b6c-a461-e4eaf3ef7937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:28.287133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:28.353562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fc486-c5c8-4b6c-a461-e4eaf3ef7937 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:28.506900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24875093-dc7c-4407-b369-658eb5aac878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:28.507864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875093-dc7c-4407-b369-658eb5aac878 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:28.508307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875093-dc7c-4407-b369-658eb5aac878 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:28.509242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24875093-dc7c-4407-b369-658eb5aac878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:28.510988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-24875093-dc7c-4407-b369-658eb5aac878 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:28.611535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:28.638570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24875093-dc7c-4407-b369-658eb5aac878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:28.640367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24875093-dc7c-4407-b369-658eb5aac878 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:28.662786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 705/1406] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:30.982335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e36cb6-c593-40ee-a38e-2cd7d896f6e8 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:30.982886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e36cb6-c593-40ee-a38e-2cd7d896f6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:30.983273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e36cb6-c593-40ee-a38e-2cd7d896f6e8 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:30.998437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 702/1407] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:18:30 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:31.002133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a6d4f-a961-4ec1-9a3d-4d3b330de372 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:31.002133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a6d4f-a961-4ec1-9a3d-4d3b330de372 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:31.002133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a6d4f-a961-4ec1-9a3d-4d3b330de372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:31.110928 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f42a6d4f-a961-4ec1-9a3d-4d3b330de372 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:31.112742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f42a6d4f-a961-4ec1-9a3d-4d3b330de372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:31.236197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 706/1408] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:18:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:31.283849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e36cb6-c593-40ee-a38e-2cd7d896f6e8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:31.283849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e6e36cb6-c593-40ee-a38e-2cd7d896f6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:31.450729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e36cb6-c593-40ee-a38e-2cd7d896f6e8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:31.547950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:31.619126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693598ff-7ac6-4025-807a-579b8721cfec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:31.620067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693598ff-7ac6-4025-807a-579b8721cfec service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:31.620543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693598ff-7ac6-4025-807a-579b8721cfec service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:31.620915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693598ff-7ac6-4025-807a-579b8721cfec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:31.623224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-693598ff-7ac6-4025-807a-579b8721cfec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:31.730354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693598ff-7ac6-4025-807a-579b8721cfec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:31.732837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693598ff-7ac6-4025-807a-579b8721cfec service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:31.765965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 703/1409] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:41.233645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=79 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:41.758635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=64 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:46.689611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbfa9d-1cc5-4dc0-a983-85f055f6de68 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:46.690243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbfa9d-1cc5-4dc0-a983-85f055f6de68 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:46.690243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbfa9d-1cc5-4dc0-a983-85f055f6de68 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:46.691801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 707/1410] 10.222.0.39 () {54 vars in 918 bytes} [Tue Jul 26 07:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:46.697103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d662c1f-36b2-4801-9ece-0f49b8a2cb4e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:46.697500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d662c1f-36b2-4801-9ece-0f49b8a2cb4e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:46.697645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d662c1f-36b2-4801-9ece-0f49b8a2cb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:46.779060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8d662c1f-36b2-4801-9ece-0f49b8a2cb4e None None] MFA Rules not processed for user `51a5093a60ef45919ab6e159adf58336`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:46.783655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d662c1f-36b2-4801-9ece-0f49b8a2cb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:46.893754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 704/1411] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Jul 26 07:18:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:46.909502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294da42d-cb48-4336-91db-441acf4bfe50 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:46.909913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294da42d-cb48-4336-91db-441acf4bfe50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:46.910199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294da42d-cb48-4336-91db-441acf4bfe50 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:46.911603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 708/1412] 10.222.0.39 () {54 vars in 918 bytes} [Tue Jul 26 07:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:46.917240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913787d-67cb-43cc-a093-5f6c78eca2a3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:46.917528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913787d-67cb-43cc-a093-5f6c78eca2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:46.917840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913787d-67cb-43cc-a093-5f6c78eca2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:47.005548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:47.009382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a913787d-67cb-43cc-a093-5f6c78eca2a3 None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:47.011427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a913787d-67cb-43cc-a093-5f6c78eca2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:47.129993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 705/1413] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Jul 26 07:18:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:18:49.618529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c652458-0601-44db-bee4-4616cdae3cd8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:49.619120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c652458-0601-44db-bee4-4616cdae3cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:49.619120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c652458-0601-44db-bee4-4616cdae3cd8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:49.620462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:49.626196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3811fcf-6a2d-47a0-89fa-9b53ba42661c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:49.626683 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3811fcf-6a2d-47a0-89fa-9b53ba42661c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:49.626800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3811fcf-6a2d-47a0-89fa-9b53ba42661c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:49.695332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a3811fcf-6a2d-47a0-89fa-9b53ba42661c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:49.696466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3811fcf-6a2d-47a0-89fa-9b53ba42661c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:49.753662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:49.798792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:49.842066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c652458-0601-44db-bee4-4616cdae3cd8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:49.843891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3c652458-0601-44db-bee4-4616cdae3cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:50.011286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c652458-0601-44db-bee4-4616cdae3cd8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:50.086856 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:50.129982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3616acfc-dbc3-441d-9137-e37e14fc8713 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:50.130622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616acfc-dbc3-441d-9137-e37e14fc8713 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:50.130901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616acfc-dbc3-441d-9137-e37e14fc8713 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:50.131171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3616acfc-dbc3-441d-9137-e37e14fc8713 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:50.132817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3616acfc-dbc3-441d-9137-e37e14fc8713 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:50.227467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616acfc-dbc3-441d-9137-e37e14fc8713 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:50.229126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3616acfc-dbc3-441d-9137-e37e14fc8713 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:50.247957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 710/1416] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:51.187784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd638d-7ee2-4f3c-bf80-b146b9478037 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:51.188367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd638d-7ee2-4f3c-bf80-b146b9478037 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:51.188367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd638d-7ee2-4f3c-bf80-b146b9478037 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:51.189708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:51.195242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40e39-b94a-4296-8a01-1056ad83ce81 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:51.195736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40e39-b94a-4296-8a01-1056ad83ce81 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:51.195736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f40e39-b94a-4296-8a01-1056ad83ce81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:51.288400 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f0f40e39-b94a-4296-8a01-1056ad83ce81 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:51.289618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f0f40e39-b94a-4296-8a01-1056ad83ce81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:51.320397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:51.396656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:51.549231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd638d-7ee2-4f3c-bf80-b146b9478037 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:51.550892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-24dd638d-7ee2-4f3c-bf80-b146b9478037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:51.715461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd638d-7ee2-4f3c-bf80-b146b9478037 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:51.757242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:51.834521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66262f-383c-4984-891b-a81f488dbb9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:51.835153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66262f-383c-4984-891b-a81f488dbb9f service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:51.835426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66262f-383c-4984-891b-a81f488dbb9f service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:51.835695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66262f-383c-4984-891b-a81f488dbb9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:51.837320 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d66262f-383c-4984-891b-a81f488dbb9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:51.930435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66262f-383c-4984-891b-a81f488dbb9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:51.932893 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66262f-383c-4984-891b-a81f488dbb9f service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:51.953947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 708/1419] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:52.845286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265664d1-ddeb-49ea-b3d9-c4dfd6dfd310 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:52.845923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265664d1-ddeb-49ea-b3d9-c4dfd6dfd310 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:52.845923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265664d1-ddeb-49ea-b3d9-c4dfd6dfd310 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:52.847504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:18:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:52.852741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92f3df-a323-4676-8a7b-a6a64e330eeb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:52.853218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92f3df-a323-4676-8a7b-a6a64e330eeb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:52.853325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92f3df-a323-4676-8a7b-a6a64e330eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:52.921090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:52.947345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-da92f3df-a323-4676-8a7b-a6a64e330eeb None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:52.948571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-da92f3df-a323-4676-8a7b-a6a64e330eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:53.061278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:18:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:18:53.214078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265664d1-ddeb-49ea-b3d9-c4dfd6dfd310 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:53.215523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-265664d1-ddeb-49ea-b3d9-c4dfd6dfd310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:53.368282 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265664d1-ddeb-49ea-b3d9-c4dfd6dfd310 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:53.381337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:53.493451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f42c8d-265d-449a-b44c-cd336ef3faf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:53.494031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f42c8d-265d-449a-b44c-cd336ef3faf4 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:53.494307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f42c8d-265d-449a-b44c-cd336ef3faf4 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:53.494573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f42c8d-265d-449a-b44c-cd336ef3faf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:53.496214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f3f42c8d-265d-449a-b44c-cd336ef3faf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:53.600892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f42c8d-265d-449a-b44c-cd336ef3faf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:53.602569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f42c8d-265d-449a-b44c-cd336ef3faf4 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:53.623123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 713/1422] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:54.506779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6c291-56c3-4bae-8a5d-a33a0a437b02 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:54.507386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6c291-56c3-4bae-8a5d-a33a0a437b02 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:54.507386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6c291-56c3-4bae-8a5d-a33a0a437b02 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:54.508688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:18:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:54.513894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca0b99-5892-49f2-9591-391ba3b3f848 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:54.514371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca0b99-5892-49f2-9591-391ba3b3f848 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:54.514477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca0b99-5892-49f2-9591-391ba3b3f848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:54.535173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:54.598515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6cca0b99-5892-49f2-9591-391ba3b3f848 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:54.599744 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6cca0b99-5892-49f2-9591-391ba3b3f848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:54.720354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:18:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:54.872073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6c291-56c3-4bae-8a5d-a33a0a437b02 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:54.873584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f4e6c291-56c3-4bae-8a5d-a33a0a437b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:54.996590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:55.039377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6c291-56c3-4bae-8a5d-a33a0a437b02 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:55.162261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24654a0a-4b6e-46cf-b771-b22f121f0c6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:55.162902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24654a0a-4b6e-46cf-b771-b22f121f0c6e service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:55.163180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24654a0a-4b6e-46cf-b771-b22f121f0c6e service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:55.163441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24654a0a-4b6e-46cf-b771-b22f121f0c6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:55.165091 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-24654a0a-4b6e-46cf-b771-b22f121f0c6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:55.248157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:55.262394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24654a0a-4b6e-46cf-b771-b22f121f0c6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:55.264009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24654a0a-4b6e-46cf-b771-b22f121f0c6e service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:55.282088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 711/1425] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:56.133271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5ed56-a92f-47ba-8780-6cc797418c7d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:56.133879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5ed56-a92f-47ba-8780-6cc797418c7d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:56.133879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b5ed56-a92f-47ba-8780-6cc797418c7d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:56.135215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:56.140376 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c588e-5642-4116-ab63-ec86d187c397 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:56.140851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c588e-5642-4116-ab63-ec86d187c397 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:56.140954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c588e-5642-4116-ab63-ec86d187c397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:56.220265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-236c588e-5642-4116-ab63-ec86d187c397 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:56.221532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-236c588e-5642-4116-ab63-ec86d187c397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:56.325687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:18:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:56.474925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5ed56-a92f-47ba-8780-6cc797418c7d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:56.476649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-06b5ed56-a92f-47ba-8780-6cc797418c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:56.567661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:56.642015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b5ed56-a92f-47ba-8780-6cc797418c7d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:56.773402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c649ac-269d-48ae-9e6d-64174507c46b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:56.774070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c649ac-269d-48ae-9e6d-64174507c46b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:56.774349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c649ac-269d-48ae-9e6d-64174507c46b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:56.774616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c649ac-269d-48ae-9e6d-64174507c46b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:56.776268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-74c649ac-269d-48ae-9e6d-64174507c46b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:56.826394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:56.870606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c649ac-269d-48ae-9e6d-64174507c46b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:56.872323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c649ac-269d-48ae-9e6d-64174507c46b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:56.891284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 716/1428] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:57.699784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7293-8525-4545-b269-aa08b7d07f67 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:57.700401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7293-8525-4545-b269-aa08b7d07f67 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:57.700401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafd7293-8525-4545-b269-aa08b7d07f67 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:57.701704 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:18:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:57.707404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4936ece-6653-4f9b-8641-84db0041b8d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:57.707972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4936ece-6653-4f9b-8641-84db0041b8d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:57.707972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4936ece-6653-4f9b-8641-84db0041b8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:57.794687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d4936ece-6653-4f9b-8641-84db0041b8d1 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:57.795194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4936ece-6653-4f9b-8641-84db0041b8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:57.904683 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:58.056944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd7293-8525-4545-b269-aa08b7d07f67 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:58.058441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eafd7293-8525-4545-b269-aa08b7d07f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:58.117692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:58.222588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafd7293-8525-4545-b269-aa08b7d07f67 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:58.342510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b044af1-aad0-456b-9d91-b0e3d0cb8439 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:58.343039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b044af1-aad0-456b-9d91-b0e3d0cb8439 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:58.343318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b044af1-aad0-456b-9d91-b0e3d0cb8439 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:58.343590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b044af1-aad0-456b-9d91-b0e3d0cb8439 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:58.345219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b044af1-aad0-456b-9d91-b0e3d0cb8439 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:58.352759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:58.448015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b044af1-aad0-456b-9d91-b0e3d0cb8439 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:18:58.449662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b044af1-aad0-456b-9d91-b0e3d0cb8439 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:18:58.469289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 714/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:18:59.270995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6093c8-4cd4-4a67-8490-8a7d09250276 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:59.271862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6093c8-4cd4-4a67-8490-8a7d09250276 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:59.271862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6093c8-4cd4-4a67-8490-8a7d09250276 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:59.272911 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:18:59.278486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad87be-44fa-4569-b688-27b88547f6ee None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:59.279218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad87be-44fa-4569-b688-27b88547f6ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:59.279218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad87be-44fa-4569-b688-27b88547f6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:59.363530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0ad87be-44fa-4569-b688-27b88547f6ee None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:18:59.364772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0ad87be-44fa-4569-b688-27b88547f6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:59.440131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:59.472416 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:18:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:18:59.622306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6093c8-4cd4-4a67-8490-8a7d09250276 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:18:59.623739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e6093c8-4cd4-4a67-8490-8a7d09250276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:18:59.642693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:59.782237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6093c8-4cd4-4a67-8490-8a7d09250276 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:18:59.886930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:18:59.917170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9d644-0abe-430c-b962-bd620060b6b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:18:59.917863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d644-0abe-430c-b962-bd620060b6b8 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:18:59.918151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d644-0abe-430c-b962-bd620060b6b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:18:59.918432 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9d644-0abe-430c-b962-bd620060b6b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:18:59.920085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dbb9d644-0abe-430c-b962-bd620060b6b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:00.034619 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9d644-0abe-430c-b962-bd620060b6b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:00.036268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9d644-0abe-430c-b962-bd620060b6b8 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:00.055119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 719/1434] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:18:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:00.877062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f48b3-5a07-4b93-9b5a-b2b658379f2d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:00.877658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f48b3-5a07-4b93-9b5a-b2b658379f2d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:00.877658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f48b3-5a07-4b93-9b5a-b2b658379f2d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:00.879023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:00.884543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d193d0-e2b3-4180-ac3e-1c6406c912bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:00.885091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d193d0-e2b3-4180-ac3e-1c6406c912bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:00.885091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d193d0-e2b3-4180-ac3e-1c6406c912bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:00.969671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b2d193d0-e2b3-4180-ac3e-1c6406c912bb None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:00.970977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2d193d0-e2b3-4180-ac3e-1c6406c912bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:01.029595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:01.095323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:01.316676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f48b3-5a07-4b93-9b5a-b2b658379f2d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:01.318265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b78f48b3-5a07-4b93-9b5a-b2b658379f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:01.472641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f48b3-5a07-4b93-9b5a-b2b658379f2d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:01.533942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:01.601864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c079a-f36c-4da0-b44a-d8e9bf07f414 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:01.602456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c079a-f36c-4da0-b44a-d8e9bf07f414 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:01.602733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c079a-f36c-4da0-b44a-d8e9bf07f414 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:01.603001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c079a-f36c-4da0-b44a-d8e9bf07f414 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:01.604809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d08c079a-f36c-4da0-b44a-d8e9bf07f414 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:01.710194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c079a-f36c-4da0-b44a-d8e9bf07f414 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:01.711797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c079a-f36c-4da0-b44a-d8e9bf07f414 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:01.734073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 717/1437] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:02.563687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfa01a-4f2e-4084-a4b8-7959d38dbd4e None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:02.564299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfa01a-4f2e-4084-a4b8-7959d38dbd4e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:02.564299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdfa01a-4f2e-4084-a4b8-7959d38dbd4e None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:02.565591 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:02.571366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cd96b-0f7d-4296-a2ad-d90c7eaef5f5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:02.571943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cd96b-0f7d-4296-a2ad-d90c7eaef5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:02.571943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83cd96b-0f7d-4296-a2ad-d90c7eaef5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:02.663231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c83cd96b-0f7d-4296-a2ad-d90c7eaef5f5 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:02.664652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c83cd96b-0f7d-4296-a2ad-d90c7eaef5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:02.667135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:02.778919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:19:02.930820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfa01a-4f2e-4084-a4b8-7959d38dbd4e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:02.932466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5cdfa01a-4f2e-4084-a4b8-7959d38dbd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:03.088246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdfa01a-4f2e-4084-a4b8-7959d38dbd4e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:03.116069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:03.207900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfaccea-9818-4914-b5df-8eb3985c800b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:03.208524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaccea-9818-4914-b5df-8eb3985c800b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:03.208801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaccea-9818-4914-b5df-8eb3985c800b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:03.209071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfaccea-9818-4914-b5df-8eb3985c800b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:03.210760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3dfaccea-9818-4914-b5df-8eb3985c800b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:03.303910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfaccea-9818-4914-b5df-8eb3985c800b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:03.305583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfaccea-9818-4914-b5df-8eb3985c800b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:03.325170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 722/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:19:05.253421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a1c2-b4cf-482e-97f8-043716aba5d2 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:05.254276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a1c2-b4cf-482e-97f8-043716aba5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:05.254276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24a1c2-b4cf-482e-97f8-043716aba5d2 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:05.255479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:05.262147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b1562-82a3-4e82-be09-74900aa0b4c6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:05.262977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b1562-82a3-4e82-be09-74900aa0b4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:05.263161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b1562-82a3-4e82-be09-74900aa0b4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:05.317004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:05.349549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b12b1562-82a3-4e82-be09-74900aa0b4c6 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:05.350902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b12b1562-82a3-4e82-be09-74900aa0b4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:05.466216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:05.531884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24a1c2-b4cf-482e-97f8-043716aba5d2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:05.534706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ce24a1c2-b4cf-482e-97f8-043716aba5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:05.721977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24a1c2-b4cf-482e-97f8-043716aba5d2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:05.779853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:05.850526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99273668-67e9-48c2-9988-cc824e3e4a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:05.851180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99273668-67e9-48c2-9988-cc824e3e4a76 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:05.851443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99273668-67e9-48c2-9988-cc824e3e4a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:05.851885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99273668-67e9-48c2-9988-cc824e3e4a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:05.853657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-99273668-67e9-48c2-9988-cc824e3e4a76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:05.972856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99273668-67e9-48c2-9988-cc824e3e4a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:05.974387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99273668-67e9-48c2-9988-cc824e3e4a76 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:05.994626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 720/1443] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:08.246739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e8c23-7825-445f-bf71-c024181e88f6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:08.247371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e8c23-7825-445f-bf71-c024181e88f6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:08.247371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e8c23-7825-445f-bf71-c024181e88f6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:08.248750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:08.256019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143abc2-249f-439b-99ab-78f049e50514 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:08.256961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143abc2-249f-439b-99ab-78f049e50514 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:08.257117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9143abc2-249f-439b-99ab-78f049e50514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:08.333968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9143abc2-249f-439b-99ab-78f049e50514 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:08.335207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9143abc2-249f-439b-99ab-78f049e50514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:08.349451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:08.445498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:08.485321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e8c23-7825-445f-bf71-c024181e88f6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:08.486942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-931e8c23-7825-445f-bf71-c024181e88f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:08.603204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e8c23-7825-445f-bf71-c024181e88f6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:08.616902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:08.683601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1aad2-9de1-4325-b09c-3e31bb594e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:08.684240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1aad2-9de1-4325-b09c-3e31bb594e97 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:08.684536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1aad2-9de1-4325-b09c-3e31bb594e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:08.684796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1aad2-9de1-4325-b09c-3e31bb594e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:08.686487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97b1aad2-9de1-4325-b09c-3e31bb594e97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:08.943886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b1aad2-9de1-4325-b09c-3e31bb594e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:08.945404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b1aad2-9de1-4325-b09c-3e31bb594e97 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:08.966130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 725/1446] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:11.325929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21725e4a-3808-492d-8275-0cb7ad77ef38 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:11.326561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21725e4a-3808-492d-8275-0cb7ad77ef38 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:11.326561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21725e4a-3808-492d-8275-0cb7ad77ef38 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:11.328460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:19:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:11.334948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241a5e8-303a-4867-accd-fa196695f9ea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:11.335423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241a5e8-303a-4867-accd-fa196695f9ea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:11.335521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241a5e8-303a-4867-accd-fa196695f9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:11.391716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:11.411309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1241a5e8-303a-4867-accd-fa196695f9ea None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:11.412526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1241a5e8-303a-4867-accd-fa196695f9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:11.511818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:11.550197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21725e4a-3808-492d-8275-0cb7ad77ef38 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:11.551779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-21725e4a-3808-492d-8275-0cb7ad77ef38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:11.662398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21725e4a-3808-492d-8275-0cb7ad77ef38 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:11.706814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:11.746548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd560641-2959-4c88-8ea6-a26219d1957c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:11.747219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd560641-2959-4c88-8ea6-a26219d1957c service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:11.747510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd560641-2959-4c88-8ea6-a26219d1957c service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:11.747801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd560641-2959-4c88-8ea6-a26219d1957c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:11.749484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bd560641-2959-4c88-8ea6-a26219d1957c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:11.880467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd560641-2959-4c88-8ea6-a26219d1957c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:11.882018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd560641-2959-4c88-8ea6-a26219d1957c service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:11.902175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 723/1449] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:14.034166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c27f5b-c4d1-4d8b-b1ff-bb45e2ed5771 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:14.034764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c27f5b-c4d1-4d8b-b1ff-bb45e2ed5771 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:14.034764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c27f5b-c4d1-4d8b-b1ff-bb45e2ed5771 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:14.036117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:14.041575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad247b-0dc6-4460-b587-c1d209a59530 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:14.042070 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad247b-0dc6-4460-b587-c1d209a59530 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:14.042173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad247b-0dc6-4460-b587-c1d209a59530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:14.112144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-08ad247b-0dc6-4460-b587-c1d209a59530 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:14.113513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-08ad247b-0dc6-4460-b587-c1d209a59530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:14.135468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:14.217435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:19:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:14.257849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c27f5b-c4d1-4d8b-b1ff-bb45e2ed5771 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:14.259451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-91c27f5b-c4d1-4d8b-b1ff-bb45e2ed5771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:14.372550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c27f5b-c4d1-4d8b-b1ff-bb45e2ed5771 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:14.394711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:14.460426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea864559-2cfe-4d2c-bc07-c0f8eee34094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:14.461061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea864559-2cfe-4d2c-bc07-c0f8eee34094 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:14.461340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea864559-2cfe-4d2c-bc07-c0f8eee34094 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:14.461608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea864559-2cfe-4d2c-bc07-c0f8eee34094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:14.463299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ea864559-2cfe-4d2c-bc07-c0f8eee34094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:14.583106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea864559-2cfe-4d2c-bc07-c0f8eee34094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:14.584889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea864559-2cfe-4d2c-bc07-c0f8eee34094 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:14.605371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 728/1452] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:16.673785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e05fa6-f3ab-4b48-9407-4dcc0040a118 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:16.674333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e05fa6-f3ab-4b48-9407-4dcc0040a118 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:16.674467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e05fa6-f3ab-4b48-9407-4dcc0040a118 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:16.675829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:19:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:16.681454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd2747-d684-4f66-8899-cd5ec826a25f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:16.681897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd2747-d684-4f66-8899-cd5ec826a25f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:16.682064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd2747-d684-4f66-8899-cd5ec826a25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:16.755546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:16.759385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3acd2747-d684-4f66-8899-cd5ec826a25f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:16.761649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3acd2747-d684-4f66-8899-cd5ec826a25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:16.865756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:19:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:16.904375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e05fa6-f3ab-4b48-9407-4dcc0040a118 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:16.905945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-25e05fa6-f3ab-4b48-9407-4dcc0040a118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:17.024221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e05fa6-f3ab-4b48-9407-4dcc0040a118 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:17.075514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:17.102056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270e81b0-a7ef-4c57-9ad4-6b872b57be97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:17.102698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e81b0-a7ef-4c57-9ad4-6b872b57be97 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:17.102982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e81b0-a7ef-4c57-9ad4-6b872b57be97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:17.103256 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270e81b0-a7ef-4c57-9ad4-6b872b57be97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:17.104901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-270e81b0-a7ef-4c57-9ad4-6b872b57be97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:17.229112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e81b0-a7ef-4c57-9ad4-6b872b57be97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:17.230600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270e81b0-a7ef-4c57-9ad4-6b872b57be97 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:17.250619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 726/1455] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:19.336889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2c957-a09d-4836-9eb0-8806d8700171 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:19.337539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2c957-a09d-4836-9eb0-8806d8700171 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:19.337539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2c957-a09d-4836-9eb0-8806d8700171 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:19.338844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:19.344774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b28c2-1dc9-44bb-9e50-7711f9540f44 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:19.345269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b28c2-1dc9-44bb-9e50-7711f9540f44 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:19.345387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b28c2-1dc9-44bb-9e50-7711f9540f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:19.415436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e09b28c2-1dc9-44bb-9e50-7711f9540f44 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:19.416668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e09b28c2-1dc9-44bb-9e50-7711f9540f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:19.449006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:19.521011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:19:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:19.559275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2c957-a09d-4836-9eb0-8806d8700171 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:19.560823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1fd2c957-a09d-4836-9eb0-8806d8700171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:19.670985 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2c957-a09d-4836-9eb0-8806d8700171 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:19.700168 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:19.749841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469bfe3-c4ba-457c-9584-c1c2fe51ca0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:19.750474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469bfe3-c4ba-457c-9584-c1c2fe51ca0e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:19.750750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469bfe3-c4ba-457c-9584-c1c2fe51ca0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:19.751016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469bfe3-c4ba-457c-9584-c1c2fe51ca0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:19.752667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3469bfe3-c4ba-457c-9584-c1c2fe51ca0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:19.869127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469bfe3-c4ba-457c-9584-c1c2fe51ca0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:19.870653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469bfe3-c4ba-457c-9584-c1c2fe51ca0e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:19.890004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 731/1458] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:22.001650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a608fd-405a-446b-a6a8-fd27a35685ba None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:22.002294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a608fd-405a-446b-a6a8-fd27a35685ba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:22.002294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a608fd-405a-446b-a6a8-fd27a35685ba None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:22.003689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:19:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:22.009174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e1cc-652b-406b-8d74-5a617d29fe7c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:22.009616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e1cc-652b-406b-8d74-5a617d29fe7c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:22.009728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311e1cc-652b-406b-8d74-5a617d29fe7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:22.077484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3311e1cc-652b-406b-8d74-5a617d29fe7c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:22.078732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3311e1cc-652b-406b-8d74-5a617d29fe7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:22.087270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:22.178358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:19:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:22.216450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a608fd-405a-446b-a6a8-fd27a35685ba None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:22.218141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-82a608fd-405a-446b-a6a8-fd27a35685ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:22.332633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a608fd-405a-446b-a6a8-fd27a35685ba None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:22.396392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:22.418373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636e1fb-f24a-4ece-8ce9-1fd24b1bd7fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:22.419009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e1fb-f24a-4ece-8ce9-1fd24b1bd7fd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:22.419294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e1fb-f24a-4ece-8ce9-1fd24b1bd7fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:22.419566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636e1fb-f24a-4ece-8ce9-1fd24b1bd7fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:22.421254 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9636e1fb-f24a-4ece-8ce9-1fd24b1bd7fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:22.555804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636e1fb-f24a-4ece-8ce9-1fd24b1bd7fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:22.557284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636e1fb-f24a-4ece-8ce9-1fd24b1bd7fd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:22.579393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 729/1461] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:24.636535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e39c78-f58f-4f2d-83a8-0489f57f1012 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:24.637179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e39c78-f58f-4f2d-83a8-0489f57f1012 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:24.637179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e39c78-f58f-4f2d-83a8-0489f57f1012 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:24.638644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:24.645134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d1546-79bb-41c2-a946-697b0058e214 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:24.645762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d1546-79bb-41c2-a946-697b0058e214 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:24.645762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d1546-79bb-41c2-a946-697b0058e214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:24.726743 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-389d1546-79bb-41c2-a946-697b0058e214 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:24.727901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-389d1546-79bb-41c2-a946-697b0058e214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:24.771357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:24.839328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:24.877120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e39c78-f58f-4f2d-83a8-0489f57f1012 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:24.878921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51e39c78-f58f-4f2d-83a8-0489f57f1012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:24.989516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e39c78-f58f-4f2d-83a8-0489f57f1012 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:25.030698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:25.069205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef041a-96d9-4139-abc2-056e67a3896f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:25.069865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef041a-96d9-4139-abc2-056e67a3896f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:25.070121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef041a-96d9-4139-abc2-056e67a3896f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:25.070413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef041a-96d9-4139-abc2-056e67a3896f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:25.072037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-23ef041a-96d9-4139-abc2-056e67a3896f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:25.184352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef041a-96d9-4139-abc2-056e67a3896f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:25.186150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef041a-96d9-4139-abc2-056e67a3896f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:25.205952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 734/1464] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:27.290419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5a273-370b-4314-98a5-fc038358ddb3 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:27.291074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5a273-370b-4314-98a5-fc038358ddb3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:27.291074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a5a273-370b-4314-98a5-fc038358ddb3 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:27.292352 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:27.298294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675d35-d818-47ed-8e78-8fa8d7479e00 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:27.298828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675d35-d818-47ed-8e78-8fa8d7479e00 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:27.298828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20675d35-d818-47ed-8e78-8fa8d7479e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:27.373754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-20675d35-d818-47ed-8e78-8fa8d7479e00 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:27.375005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20675d35-d818-47ed-8e78-8fa8d7479e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:27.393800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:27.477876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:19:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:27.538046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5a273-370b-4314-98a5-fc038358ddb3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:27.539616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a2a5a273-370b-4314-98a5-fc038358ddb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:27.660005 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a5a273-370b-4314-98a5-fc038358ddb3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:27.757980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:27.770593 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e123c-0c4b-4dec-8440-02cdfc6ae498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:27.771694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e123c-0c4b-4dec-8440-02cdfc6ae498 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:27.772200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e123c-0c4b-4dec-8440-02cdfc6ae498 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:27.772694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e123c-0c4b-4dec-8440-02cdfc6ae498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:27.775892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e35e123c-0c4b-4dec-8440-02cdfc6ae498 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:27.907574 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e123c-0c4b-4dec-8440-02cdfc6ae498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:27.909155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e123c-0c4b-4dec-8440-02cdfc6ae498 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:27.931162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 732/1467] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:29.983195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da66d6b-8e50-4e2f-a084-8a241d5850a9 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:29.983778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da66d6b-8e50-4e2f-a084-8a241d5850a9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:29.983778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da66d6b-8e50-4e2f-a084-8a241d5850a9 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:29.985124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:29.990932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e90ea-9572-4373-b092-da89dee8c2d7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:29.991745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e90ea-9572-4373-b092-da89dee8c2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:29.991942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379e90ea-9572-4373-b092-da89dee8c2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:30.061848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-379e90ea-9572-4373-b092-da89dee8c2d7 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:30.063074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-379e90ea-9572-4373-b092-da89dee8c2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:30.113588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:30.166642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:30.204069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da66d6b-8e50-4e2f-a084-8a241d5850a9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:30.205627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4da66d6b-8e50-4e2f-a084-8a241d5850a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:30.312679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da66d6b-8e50-4e2f-a084-8a241d5850a9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:30.362487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:30.392010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2bb157-f992-4b1c-9445-6f31ae10c1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:30.392646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb157-f992-4b1c-9445-6f31ae10c1fd service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:30.392979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb157-f992-4b1c-9445-6f31ae10c1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:30.393362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2bb157-f992-4b1c-9445-6f31ae10c1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:30.395155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8d2bb157-f992-4b1c-9445-6f31ae10c1fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:30.513441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bb157-f992-4b1c-9445-6f31ae10c1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:30.515087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2bb157-f992-4b1c-9445-6f31ae10c1fd service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:30.533782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 737/1470] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:40.164677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=22 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:40.531495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=66 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:50.285533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710cb76-d30c-40f2-b0c6-8dc1e50e5e1a None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:50.286426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710cb76-d30c-40f2-b0c6-8dc1e50e5e1a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:50.286426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710cb76-d30c-40f2-b0c6-8dc1e50e5e1a None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:50.287556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 734/1471] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:50.295957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fc9c1-38e5-4530-92a7-fd820221bed6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:50.296504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fc9c1-38e5-4530-92a7-fd820221bed6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:50.296614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fc9c1-38e5-4530-92a7-fd820221bed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:50.370079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-242fc9c1-38e5-4530-92a7-fd820221bed6 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:50.371325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-242fc9c1-38e5-4530-92a7-fd820221bed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:50.479913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 738/1472] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:50.569520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c6d90-d427-44b6-9947-8e6e3e7f8677 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:50.569809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c6d90-d427-44b6-9947-8e6e3e7f8677 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:50.570082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c6d90-d427-44b6-9947-8e6e3e7f8677 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:50.571480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 735/1473] 10.222.0.39 () {54 vars in 966 bytes} [Tue Jul 26 07:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:50.576824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6956890-ce44-4d4f-b9b0-bf11fd949b6b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:50.577128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6956890-ce44-4d4f-b9b0-bf11fd949b6b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:50.577407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6956890-ce44-4d4f-b9b0-bf11fd949b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:50.674996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d6956890-ce44-4d4f-b9b0-bf11fd949b6b None None] MFA Rules not processed for user `a3c6943686c843e588df14f7c6dffb8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:50.676232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d6956890-ce44-4d4f-b9b0-bf11fd949b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:50.697806 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:50.759764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 739/1474] 10.222.0.39 () {58 vars in 1063 bytes} [Tue Jul 26 07:19:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:50.768265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c6d90-d427-44b6-9947-8e6e3e7f8677 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:50.770072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-347c6d90-d427-44b6-9947-8e6e3e7f8677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:50.877209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c6d90-d427-44b6-9947-8e6e3e7f8677 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:50.961124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019bec2-5d41-443b-a0ab-052f940ce9eb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c6943686c843e588df14f7c6dffb8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:50.961668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019bec2-5d41-443b-a0ab-052f940ce9eb service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:50.961976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019bec2-5d41-443b-a0ab-052f940ce9eb service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:50.962244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019bec2-5d41-443b-a0ab-052f940ce9eb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:50.964227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c019bec2-5d41-443b-a0ab-052f940ce9eb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:51.029528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:51.089917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019bec2-5d41-443b-a0ab-052f940ce9eb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:51.091581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019bec2-5d41-443b-a0ab-052f940ce9eb service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:51.112538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 736/1475] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:53.250630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcf99b-e1f1-4f3b-9a2a-103fa1b49fcf None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:53.251523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcf99b-e1f1-4f3b-9a2a-103fa1b49fcf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:53.251523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bcf99b-e1f1-4f3b-9a2a-103fa1b49fcf None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:53.252746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 740/1476] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:53.258917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5c6e3-5377-4839-b042-3c44e2d44c42 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:53.259745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5c6e3-5377-4839-b042-3c44e2d44c42 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:53.259745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5c6e3-5377-4839-b042-3c44e2d44c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:53.345955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-75b5c6e3-5377-4839-b042-3c44e2d44c42 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:53.347825 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-75b5c6e3-5377-4839-b042-3c44e2d44c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:53.508792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 737/1477] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:53.548187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcf99b-e1f1-4f3b-9a2a-103fa1b49fcf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:53.551057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7bcf99b-e1f1-4f3b-9a2a-103fa1b49fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:53.763711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bcf99b-e1f1-4f3b-9a2a-103fa1b49fcf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:53.822719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:53.881281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea55b7a-0f56-4a8a-969c-abc61624eaeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:53.881969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea55b7a-0f56-4a8a-969c-abc61624eaeb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:53.882252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea55b7a-0f56-4a8a-969c-abc61624eaeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:53.882523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea55b7a-0f56-4a8a-969c-abc61624eaeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:53.883233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea55b7a-0f56-4a8a-969c-abc61624eaeb admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:53.885030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-2ea55b7a-0f56-4a8a-969c-abc61624eaeb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:19:53.885030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 26 07:19:53.885030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: Jul 26 07:19:53.886263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea55b7a-0f56-4a8a-969c-abc61624eaeb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:53.907075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 741/1478] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Jul 26 07:19:53 2022] GET /identity/v3/projects => generated 2553 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:19:56.041932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b71536-6fe9-4033-bdb3-58f93e40053b None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:56.042580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b71536-6fe9-4033-bdb3-58f93e40053b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:56.042580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b71536-6fe9-4033-bdb3-58f93e40053b None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:56.044155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 738/1479] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:56.050692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb63e70-2b23-4fb6-bc4e-d5c1328891d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:56.051574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb63e70-2b23-4fb6-bc4e-d5c1328891d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:56.051574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb63e70-2b23-4fb6-bc4e-d5c1328891d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:56.131349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fdb63e70-2b23-4fb6-bc4e-d5c1328891d1 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:56.132749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fdb63e70-2b23-4fb6-bc4e-d5c1328891d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:56.286464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 742/1480] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:19:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:19:56.332170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b71536-6fe9-4033-bdb3-58f93e40053b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:56.333901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-91b71536-6fe9-4033-bdb3-58f93e40053b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:56.344910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:56.626841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b71536-6fe9-4033-bdb3-58f93e40053b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:56.716014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:56.759086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f3ca6-a3d4-477a-9f78-15b1519d162f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:56.759819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f3ca6-a3d4-477a-9f78-15b1519d162f service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:56.760136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f3ca6-a3d4-477a-9f78-15b1519d162f service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:56.760441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f3ca6-a3d4-477a-9f78-15b1519d162f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:56.762204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b89f3ca6-a3d4-477a-9f78-15b1519d162f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:56.885587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f3ca6-a3d4-477a-9f78-15b1519d162f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:56.887243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89f3ca6-a3d4-477a-9f78-15b1519d162f service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:56.911031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 739/1481] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 580 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:19:59.130782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cb991-4b24-4c63-b433-48d8b7fb6f06 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:59.131424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cb991-4b24-4c63-b433-48d8b7fb6f06 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:59.131424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2cb991-4b24-4c63-b433-48d8b7fb6f06 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:59.132712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 743/1482] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:19:59.138539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f63836-3997-42e3-b1b0-9e7dd6c1c92b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:59.139067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f63836-3997-42e3-b1b0-9e7dd6c1c92b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:59.139067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f63836-3997-42e3-b1b0-9e7dd6c1c92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:59.208333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-16f63836-3997-42e3-b1b0-9e7dd6c1c92b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:19:59.209577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-16f63836-3997-42e3-b1b0-9e7dd6c1c92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:59.257792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:59.312997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 740/1483] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:19:59.351179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cb991-4b24-4c63-b433-48d8b7fb6f06 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:19:59.352600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0f2cb991-4b24-4c63-b433-48d8b7fb6f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:59.379037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:59.471264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2cb991-4b24-4c63-b433-48d8b7fb6f06 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:19:59.552993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd4ed0-f7bb-4cf3-b7fc-ad33b7352077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:19:59.553633 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd4ed0-f7bb-4cf3-b7fc-ad33b7352077 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:19:59.553944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd4ed0-f7bb-4cf3-b7fc-ad33b7352077 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:19:59.554226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd4ed0-f7bb-4cf3-b7fc-ad33b7352077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:19:59.555952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0bd4ed0-f7bb-4cf3-b7fc-ad33b7352077 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:19:59.640284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:19:59.686261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd4ed0-f7bb-4cf3-b7fc-ad33b7352077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:19:59.688384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd4ed0-f7bb-4cf3-b7fc-ad33b7352077 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:19:59.708577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 744/1484] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:20:01.777635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671a917-869a-48fd-adca-7f79aaf23d9f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:20:01.778264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671a917-869a-48fd-adca-7f79aaf23d9f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:20:01.778264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671a917-869a-48fd-adca-7f79aaf23d9f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:20:01.779554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 741/1485] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:20:01.785821 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c374d-0876-4eda-8c8b-82c67fa84289 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:20:01.786493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c374d-0876-4eda-8c8b-82c67fa84289 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:20:01.786493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c374d-0876-4eda-8c8b-82c67fa84289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:20:01.865602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-174c374d-0876-4eda-8c8b-82c67fa84289 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:20:01.866892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-174c374d-0876-4eda-8c8b-82c67fa84289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:20:01.967800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 745/1486] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:20:02.006274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671a917-869a-48fd-adca-7f79aaf23d9f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:20:02.007839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0671a917-869a-48fd-adca-7f79aaf23d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:20:02.119256 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671a917-869a-48fd-adca-7f79aaf23d9f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:20:02.194871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:20:02.196740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16346a3-9dbf-4e14-8611-53c009cd2d47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:20:02.197432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16346a3-9dbf-4e14-8611-53c009cd2d47 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:20:02.197841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16346a3-9dbf-4e14-8611-53c009cd2d47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:20:02.198158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16346a3-9dbf-4e14-8611-53c009cd2d47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:20:02.200222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e16346a3-9dbf-4e14-8611-53c009cd2d47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:20:02.325619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16346a3-9dbf-4e14-8611-53c009cd2d47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:20:02.327105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16346a3-9dbf-4e14-8611-53c009cd2d47 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:20:02.346761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 742/1487] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:20:04.554468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ecfc8-c07f-4408-9d8c-781a6fc814dc None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:20:04.555140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ecfc8-c07f-4408-9d8c-781a6fc814dc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:20:04.555140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ecfc8-c07f-4408-9d8c-781a6fc814dc None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:20:04.556385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 746/1488] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:20:04.564689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5840786-8859-45f3-ac72-4c6da9819b6d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:20:04.565304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5840786-8859-45f3-ac72-4c6da9819b6d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:20:04.565304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5840786-8859-45f3-ac72-4c6da9819b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:20:04.646574 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a5840786-8859-45f3-ac72-4c6da9819b6d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:20:04.648030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a5840786-8859-45f3-ac72-4c6da9819b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:20:04.716502 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:20:04.763461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 743/1489] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:20:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:20:04.803903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ecfc8-c07f-4408-9d8c-781a6fc814dc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:20:04.805555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-705ecfc8-c07f-4408-9d8c-781a6fc814dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:20:04.846097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:20:04.918621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ecfc8-c07f-4408-9d8c-781a6fc814dc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:20:04.996304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d27327-0325-4102-8311-75227072a597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:20:04.997026 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27327-0325-4102-8311-75227072a597 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:20:04.997344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27327-0325-4102-8311-75227072a597 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:20:04.997651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d27327-0325-4102-8311-75227072a597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:20:04.999480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-99d27327-0325-4102-8311-75227072a597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:20:05.088047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:20:05.120013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d27327-0325-4102-8311-75227072a597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:20:05.121457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d27327-0325-4102-8311-75227072a597 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:20:05.140570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 747/1490] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:20:07.540231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128afaf7-1018-49e5-995f-c975bdd80122 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:20:07.540829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128afaf7-1018-49e5-995f-c975bdd80122 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:20:07.540829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128afaf7-1018-49e5-995f-c975bdd80122 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:20:07.542172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 744/1491] 10.222.0.39 () {54 vars in 926 bytes} [Tue Jul 26 07:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:20:07.548157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebecd5f-15c7-44a3-b18d-1e1879ec2951 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:20:07.548729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebecd5f-15c7-44a3-b18d-1e1879ec2951 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:20:07.548729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebecd5f-15c7-44a3-b18d-1e1879ec2951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:20:07.626197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cebecd5f-15c7-44a3-b18d-1e1879ec2951 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:20:07.628158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cebecd5f-15c7-44a3-b18d-1e1879ec2951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:20:07.742648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 748/1492] 10.222.0.39 () {58 vars in 1023 bytes} [Tue Jul 26 07:20:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:20:07.782727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128afaf7-1018-49e5-995f-c975bdd80122 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:20:07.784260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-128afaf7-1018-49e5-995f-c975bdd80122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:20:07.898128 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128afaf7-1018-49e5-995f-c975bdd80122 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:20:07.974336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad64056-da74-4e9c-aed9-816352b0224a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:20:07.974961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad64056-da74-4e9c-aed9-816352b0224a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:20:07.975242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad64056-da74-4e9c-aed9-816352b0224a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:20:07.975515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad64056-da74-4e9c-aed9-816352b0224a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:20:07.977158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ad64056-da74-4e9c-aed9-816352b0224a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:20:07.987044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:20:08.094907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad64056-da74-4e9c-aed9-816352b0224a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:20:08.096388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad64056-da74-4e9c-aed9-816352b0224a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:20:08.116184 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 745/1493] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:20:17.740492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:20:18.113661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=50 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:43.471922 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e99fa-6a83-4072-ab7f-a948fd16bd12 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:43.472548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e99fa-6a83-4072-ab7f-a948fd16bd12 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:43.472548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e99fa-6a83-4072-ab7f-a948fd16bd12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:43.552184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-346e99fa-6a83-4072-ab7f-a948fd16bd12 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:21:43.553457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-346e99fa-6a83-4072-ab7f-a948fd16bd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:43.655349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 749/1494] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:21:43.663963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d7c92-99a6-4863-8aba-8910875f097e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:43.664486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d7c92-99a6-4863-8aba-8910875f097e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:43.664486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d7c92-99a6-4863-8aba-8910875f097e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:43.745787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-629d7c92-99a6-4863-8aba-8910875f097e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:21:43.746972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-629d7c92-99a6-4863-8aba-8910875f097e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:43.847299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 746/1495] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:21:43.852654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e99fa-6a83-4072-ab7f-a948fd16bd12 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:21:43.854067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-346e99fa-6a83-4072-ab7f-a948fd16bd12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:43.866097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e99fa-6a83-4072-ab7f-a948fd16bd12 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:21:43.929690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:43.964711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:21:43.965359 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:43.965663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:43.965942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:43.966700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:21:43.966927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:21:43.968150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:21:43.968150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 26 07:21:43.968150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: Jul 26 07:21:43.969049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:21:43.981421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 750/1496] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:21:43.989937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d7c92-99a6-4863-8aba-8910875f097e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:21:43.999072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629d7c92-99a6-4863-8aba-8910875f097e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:21:44.060728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:44.095249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56960d7-5e2c-4b57-98fd-2cfadf832a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:21:44.095868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56960d7-5e2c-4b57-98fd-2cfadf832a79 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:44.096139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56960d7-5e2c-4b57-98fd-2cfadf832a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:44.096401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56960d7-5e2c-4b57-98fd-2cfadf832a79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:44.097065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56960d7-5e2c-4b57-98fd-2cfadf832a79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:21:44.098826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-a56960d7-5e2c-4b57-98fd-2cfadf832a79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:21:44.098826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 26 07:21:44.098826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: Jul 26 07:21:44.100023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56960d7-5e2c-4b57-98fd-2cfadf832a79 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:21:44.160398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 747/1497] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:21:43 2022] POST /identity/v3/projects => generated 377 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:21:44.165067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:21:44.166650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:44.202407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26148a79-8779-4114-8b7a-d16f0f1d554e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:21:44.300004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96862164-f322-4d13-b08a-8867e81df52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:21:44.300623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96862164-f322-4d13-b08a-8867e81df52e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:44.300905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96862164-f322-4d13-b08a-8867e81df52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:44.301180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96862164-f322-4d13-b08a-8867e81df52e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:44.301885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96862164-f322-4d13-b08a-8867e81df52e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:21:44.302904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-96862164-f322-4d13-b08a-8867e81df52e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:21:44.302904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 26 07:21:44.302904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: Jul 26 07:21:44.304130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96862164-f322-4d13-b08a-8867e81df52e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:21:44.349477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 751/1498] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:21:44 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:21:44.354275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56960d7-5e2c-4b57-98fd-2cfadf832a79 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:21:44.355834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a56960d7-5e2c-4b57-98fd-2cfadf832a79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:44.420646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56960d7-5e2c-4b57-98fd-2cfadf832a79 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:21:44.508643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:44.521325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec8d69-3ddd-4741-82f8-3d557401f118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:21:44.521989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec8d69-3ddd-4741-82f8-3d557401f118 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:44.522269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec8d69-3ddd-4741-82f8-3d557401f118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:44.522544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec8d69-3ddd-4741-82f8-3d557401f118 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:44.523240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec8d69-3ddd-4741-82f8-3d557401f118 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:21:44.524250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-7bec8d69-3ddd-4741-82f8-3d557401f118 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:21:44.524250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 26 07:21:44.524250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: Jul 26 07:21:44.525137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec8d69-3ddd-4741-82f8-3d557401f118 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:21:44.537910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 748/1499] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:21:44.542276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96862164-f322-4d13-b08a-8867e81df52e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:21:44.543869 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-96862164-f322-4d13-b08a-8867e81df52e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:44.555700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96862164-f322-4d13-b08a-8867e81df52e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:21:44.582390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:44.654774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfdda06-c8e1-4902-b114-a1fd307fcf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:21:44.655438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdda06-c8e1-4902-b114-a1fd307fcf99 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:44.655726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdda06-c8e1-4902-b114-a1fd307fcf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:44.655998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfdda06-c8e1-4902-b114-a1fd307fcf99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:44.656708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdda06-c8e1-4902-b114-a1fd307fcf99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:21:44.657732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-5bfdda06-c8e1-4902-b114-a1fd307fcf99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:21:44.657732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 26 07:21:44.657732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: Jul 26 07:21:44.658676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfdda06-c8e1-4902-b114-a1fd307fcf99 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:21:44.671936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 752/1500] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:21:44.676565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec8d69-3ddd-4741-82f8-3d557401f118 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:21:44.678144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7bec8d69-3ddd-4741-82f8-3d557401f118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:44.712265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec8d69-3ddd-4741-82f8-3d557401f118 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:21:44.811700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836c5d6-1182-498a-bf60-b1debe59693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:21:44.812297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836c5d6-1182-498a-bf60-b1debe59693c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:44.812572 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836c5d6-1182-498a-bf60-b1debe59693c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:44.812841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836c5d6-1182-498a-bf60-b1debe59693c admin admin] PATH_INFO: `/v3/projects/ac1eb9d0557244308ddd0f303921e2d5/users/47cd7b5c641746aa80b823d11b72d3b1/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:44.837148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836c5d6-1182-498a-bf60-b1debe59693c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac1eb9d0557244308ddd0f303921e2d5, user_id=47cd7b5c641746aa80b823d11b72d3b1, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:21:44.838222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-3836c5d6-1182-498a-bf60-b1debe59693c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:21:44.838222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 26 07:21:44.838222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: Jul 26 07:21:44.840243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836c5d6-1182-498a-bf60-b1debe59693c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:21:44.866519 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 749/1501] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:21:44 2022] PUT /identity/v3/projects/ac1eb9d0557244308ddd0f303921e2d5/users/47cd7b5c641746aa80b823d11b72d3b1/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:21:44.879115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac200685-71ea-47f9-8013-8ee6324d170f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:44.879365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac200685-71ea-47f9-8013-8ee6324d170f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:44.879680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac200685-71ea-47f9-8013-8ee6324d170f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:44.939112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ac200685-71ea-47f9-8013-8ee6324d170f None None] MFA Rules not processed for user `47cd7b5c641746aa80b823d11b72d3b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:21:44.940351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ac200685-71ea-47f9-8013-8ee6324d170f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:45.020727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 753/1502] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:21:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:21:45.035710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec750227-12ac-4197-abb4-c65a6aeb03a9 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:45.035995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec750227-12ac-4197-abb4-c65a6aeb03a9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:45.036279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec750227-12ac-4197-abb4-c65a6aeb03a9 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:45.037672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 750/1503] 10.222.0.39 () {54 vars in 964 bytes} [Tue Jul 26 07:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:21:45.042815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909063b9-36f0-499c-93c0-28fbfaaf7d80 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:45.043110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909063b9-36f0-499c-93c0-28fbfaaf7d80 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:45.043384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909063b9-36f0-499c-93c0-28fbfaaf7d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:45.076368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:45.122047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-909063b9-36f0-499c-93c0-28fbfaaf7d80 None None] MFA Rules not processed for user `becff3f1dacb43c8a014c95456e86bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:21:45.123286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-909063b9-36f0-499c-93c0-28fbfaaf7d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:45.201920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 754/1504] 10.222.0.39 () {58 vars in 1061 bytes} [Tue Jul 26 07:21:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:21:45.210173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec750227-12ac-4197-abb4-c65a6aeb03a9 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:21:45.211696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ec750227-12ac-4197-abb4-c65a6aeb03a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:45.266735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:45.320575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec750227-12ac-4197-abb4-c65a6aeb03a9 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:21:45.404032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863b15b-e879-4d7e-a6bf-e52f10766a7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:21:45.404544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863b15b-e879-4d7e-a6bf-e52f10766a7f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:45.404863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863b15b-e879-4d7e-a6bf-e52f10766a7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:45.405072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863b15b-e879-4d7e-a6bf-e52f10766a7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:45.406738 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e863b15b-e879-4d7e-a6bf-e52f10766a7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:45.514000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863b15b-e879-4d7e-a6bf-e52f10766a7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:21:45.515472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863b15b-e879-4d7e-a6bf-e52f10766a7f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:21:45.519048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:45.534280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 751/1505] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:21:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:21:45.583418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236c7ce-a258-4241-ac5f-cca028e3c9fd None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:45.583911 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236c7ce-a258-4241-ac5f-cca028e3c9fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:45.584013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6236c7ce-a258-4241-ac5f-cca028e3c9fd None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:45.585287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 755/1506] 10.222.0.39 () {52 vars in 830 bytes} [Tue Jul 26 07:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:21:45.589211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863b15b-e879-4d7e-a6bf-e52f10766a7f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:21:45.590493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e863b15b-e879-4d7e-a6bf-e52f10766a7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:45.692511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863b15b-e879-4d7e-a6bf-e52f10766a7f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:21:45.790245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:21:45.790894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:45.791170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:45.791436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] PATH_INFO: `/v3/users/47cd7b5c641746aa80b823d11b72d3b1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:45.805149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47cd7b5c641746aa80b823d11b72d3b1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:21:45.806208 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:21:45.806208 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 26 07:21:45.806208 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: Jul 26 07:21:45.807429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:21:45.817852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:45.873344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47cd7b5c641746aa80b823d11b72d3b1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:21:45.873908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47cd7b5c641746aa80b823d11b72d3b1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:21:45.891913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47cd7b5c641746aa80b823d11b72d3b1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:21:45.904171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47cd7b5c641746aa80b823d11b72d3b1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:21:45.914562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47cd7b5c641746aa80b823d11b72d3b1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:21:45.920323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 752/1507] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:21:45 2022] DELETE /identity/v3/users/47cd7b5c641746aa80b823d11b72d3b1 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:21:45.925088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039938d-cfb4-43ea-a32c-33a77d031f81 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:45.925374 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039938d-cfb4-43ea-a32c-33a77d031f81 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:45.925648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e039938d-cfb4-43ea-a32c-33a77d031f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:45.982746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e039938d-cfb4-43ea-a32c-33a77d031f81 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:21:45.983860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e039938d-cfb4-43ea-a32c-33a77d031f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:46.061849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:46.074886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 756/1508] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:21:46.084811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:21:46.086093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:46.195528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad2d7b9-5980-42c2-818c-f03f20eb6977 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:21:46.252069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:46.273694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca56759-3d88-44c5-b6ca-ca9566206545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:21:46.274356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca56759-3d88-44c5-b6ca-ca9566206545 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:46.274636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca56759-3d88-44c5-b6ca-ca9566206545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:46.274908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca56759-3d88-44c5-b6ca-ca9566206545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:46.276534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0ca56759-3d88-44c5-b6ca-ca9566206545 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:46.390577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca56759-3d88-44c5-b6ca-ca9566206545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:21:46.392043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca56759-3d88-44c5-b6ca-ca9566206545 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:21:46.412184 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 753/1509] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:21:46.747941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039938d-cfb4-43ea-a32c-33a77d031f81 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:21:46.749185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e039938d-cfb4-43ea-a32c-33a77d031f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:21:46.772257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e039938d-cfb4-43ea-a32c-33a77d031f81 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:21:46.873770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:21:46.874451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:21:46.874735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:21:46.875012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] PATH_INFO: `/v3/projects/ac1eb9d0557244308ddd0f303921e2d5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:21:46.885062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac1eb9d0557244308ddd0f303921e2d5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:21:46.886130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:21:46.886130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 26 07:21:46.886130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: Jul 26 07:21:46.887342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:21:46.949061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac1eb9d0557244308ddd0f303921e2d5', 'request_id': 'req-9c0288c5-44d9-42b5-85f3-7d2b224f1183'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:21:46.955955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac1eb9d0557244308ddd0f303921e2d5', 'request_id': 'req-9c0288c5-44d9-42b5-85f3-7d2b224f1183'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:21:46.965724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac1eb9d0557244308ddd0f303921e2d5', 'request_id': 'req-9c0288c5-44d9-42b5-85f3-7d2b224f1183'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:21:46.997081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] The token cache is being invalidate because project ac1eb9d0557244308ddd0f303921e2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:21:46.997355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:21:46.997879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9c0288c5-44d9-42b5-85f3-7d2b224f1183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:21:46.999246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 757/1510] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:21:46 2022] DELETE /identity/v3/projects/ac1eb9d0557244308ddd0f303921e2d5 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:21:56.409623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=62 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:21:56.995302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:23:24.119783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c14b8d-03ee-47b1-8fe0-f923b8ca3533 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:24.120439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c14b8d-03ee-47b1-8fe0-f923b8ca3533 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:24.120439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c14b8d-03ee-47b1-8fe0-f923b8ca3533 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:24.121896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:23:24.128585 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593611a1-f436-4b85-81d6-d920caecea75 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:24.129025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593611a1-f436-4b85-81d6-d920caecea75 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:24.129129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593611a1-f436-4b85-81d6-d920caecea75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:24.220543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-593611a1-f436-4b85-81d6-d920caecea75 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:23:24.221776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-593611a1-f436-4b85-81d6-d920caecea75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:24.328802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:23:24.482582 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c14b8d-03ee-47b1-8fe0-f923b8ca3533 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:23:24.484912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c5c14b8d-03ee-47b1-8fe0-f923b8ca3533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:24.614840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c14b8d-03ee-47b1-8fe0-f923b8ca3533 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:23:24.621222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975d3d54-a76b-4bdc-831e-9e7b17743db9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:23:24.621891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d3d54-a76b-4bdc-831e-9e7b17743db9 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:24.622176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d3d54-a76b-4bdc-831e-9e7b17743db9 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:24.622450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975d3d54-a76b-4bdc-831e-9e7b17743db9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:24.631126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d3d54-a76b-4bdc-831e-9e7b17743db9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:23:24.632736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975d3d54-a76b-4bdc-831e-9e7b17743db9 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:23:24.642773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 755/1513] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:23:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:23:25.475499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8e70b-7dfb-4349-9a5f-2554efd9e7a6 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:25.476092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8e70b-7dfb-4349-9a5f-2554efd9e7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:25.476092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8e70b-7dfb-4349-9a5f-2554efd9e7a6 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:25.477666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:23:25.483584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efe198-db0a-4f4e-9f6a-08e758a800b3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:25.484232 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efe198-db0a-4f4e-9f6a-08e758a800b3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:25.484232 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7efe198-db0a-4f4e-9f6a-08e758a800b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:25.504667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c7efe198-db0a-4f4e-9f6a-08e758a800b3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:23:25.505955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c7efe198-db0a-4f4e-9f6a-08e758a800b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:25.514585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:23:25.667158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8e70b-7dfb-4349-9a5f-2554efd9e7a6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:23:25.681685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8e70b-7dfb-4349-9a5f-2554efd9e7a6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:23:25.687130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e7022-3d3d-49f0-8c54-2f7c7af18578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:23:25.687745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e7022-3d3d-49f0-8c54-2f7c7af18578 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:25.688023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e7022-3d3d-49f0-8c54-2f7c7af18578 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:25.688294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e7022-3d3d-49f0-8c54-2f7c7af18578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:25.696737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e7022-3d3d-49f0-8c54-2f7c7af18578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:23:25.698510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e7022-3d3d-49f0-8c54-2f7c7af18578 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:23:25.708902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 760/1516] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:23:26.202195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa15d24-c5a9-47d4-b081-c1b8d03537da None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:26.202490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa15d24-c5a9-47d4-b081-c1b8d03537da None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:26.202742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa15d24-c5a9-47d4-b081-c1b8d03537da None None] PATH_INFO: `/v3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:26.204060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 757/1517] 10.222.0.43 () {54 vars in 925 bytes} [Tue Jul 26 07:23:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:23:26.216859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e04c18-6c5c-49ca-b617-f3893d439506 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:26.217144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e04c18-6c5c-49ca-b617-f3893d439506 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:26.217422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e04c18-6c5c-49ca-b617-f3893d439506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:26.283252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-33e04c18-6c5c-49ca-b617-f3893d439506 None None] MFA Rules not processed for user `51a5093a60ef45919ab6e159adf58336`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:23:26.284472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-33e04c18-6c5c-49ca-b617-f3893d439506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:26.304152 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:23:26.368191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 761/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Tue Jul 26 07:23:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:23:30.162064 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316905e-4fec-47d1-b157-b6281e727033 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:30.162623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316905e-4fec-47d1-b157-b6281e727033 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:30.162623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316905e-4fec-47d1-b157-b6281e727033 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:30.163878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 758/1519] 10.222.0.39 () {54 vars in 961 bytes} [Tue Jul 26 07:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:23:30.169000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c150b-30c4-433b-b8a2-5952c71732c3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:30.169443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c150b-30c4-433b-b8a2-5952c71732c3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:30.169538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c150b-30c4-433b-b8a2-5952c71732c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:30.191861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bd7c150b-30c4-433b-b8a2-5952c71732c3 None None] MFA Rules not processed for user `51a5093a60ef45919ab6e159adf58336`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:23:30.193075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bd7c150b-30c4-433b-b8a2-5952c71732c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:30.202567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 762/1520] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Jul 26 07:23:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:23:30.210547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316905e-4fec-47d1-b157-b6281e727033 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:23:30.225536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316905e-4fec-47d1-b157-b6281e727033 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:23:30.230701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a921c-01cb-4f1a-bd6b-4eba8a6a7dab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:23:30.231338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a921c-01cb-4f1a-bd6b-4eba8a6a7dab service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:30.231722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a921c-01cb-4f1a-bd6b-4eba8a6a7dab service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:30.232019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a921c-01cb-4f1a-bd6b-4eba8a6a7dab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:30.240784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a921c-01cb-4f1a-bd6b-4eba8a6a7dab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:23:30.242441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a921c-01cb-4f1a-bd6b-4eba8a6a7dab service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:23:30.252530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 759/1521] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:23:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:23:36.944307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316bc88-9249-46e0-a729-23210f28541f None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:36.944984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316bc88-9249-46e0-a729-23210f28541f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:36.944984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316bc88-9249-46e0-a729-23210f28541f None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:36.946284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:23:36.951695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2db94-1c34-400e-a8e0-90cc8803b80f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:36.952414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2db94-1c34-400e-a8e0-90cc8803b80f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:36.952590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2db94-1c34-400e-a8e0-90cc8803b80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:36.973758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-83e2db94-1c34-400e-a8e0-90cc8803b80f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:23:36.974971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-83e2db94-1c34-400e-a8e0-90cc8803b80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:36.983354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:23:37.141109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316bc88-9249-46e0-a729-23210f28541f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:23:37.157221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316bc88-9249-46e0-a729-23210f28541f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:23:37.163222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0746ddd-df70-4999-a8cf-17daa28e3905 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:23:37.163843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0746ddd-df70-4999-a8cf-17daa28e3905 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:37.164119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0746ddd-df70-4999-a8cf-17daa28e3905 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:37.164393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0746ddd-df70-4999-a8cf-17daa28e3905 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:37.173399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0746ddd-df70-4999-a8cf-17daa28e3905 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:23:37.175078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0746ddd-df70-4999-a8cf-17daa28e3905 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:23:37.185146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 764/1524] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:23:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:23:37.998550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12aa29b-2348-4f89-a799-4c1140792c45 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:37.999164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12aa29b-2348-4f89-a799-4c1140792c45 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:37.999164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12aa29b-2348-4f89-a799-4c1140792c45 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:38.000448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 26 07:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:23:38.005540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74278e4-da0f-4df9-8a8c-299d9500d9a5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:38.006014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74278e4-da0f-4df9-8a8c-299d9500d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:38.006118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74278e4-da0f-4df9-8a8c-299d9500d9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:38.026999 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d74278e4-da0f-4df9-8a8c-299d9500d9a5 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:23:38.028209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d74278e4-da0f-4df9-8a8c-299d9500d9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:38.037564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 26 07:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:23:38.188418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12aa29b-2348-4f89-a799-4c1140792c45 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:23:38.203991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12aa29b-2348-4f89-a799-4c1140792c45 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:23:38.209628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbb97c-16a9-4936-bc43-d54fda613c06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:23:38.210280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbb97c-16a9-4936-bc43-d54fda613c06 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:38.210560 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbb97c-16a9-4936-bc43-d54fda613c06 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:38.210833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbb97c-16a9-4936-bc43-d54fda613c06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:38.220051 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbb97c-16a9-4936-bc43-d54fda613c06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:23:38.221687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbb97c-16a9-4936-bc43-d54fda613c06 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:23:38.232624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 762/1527] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:23:42.051232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b977e1-1d1d-4ac2-8445-eb2cd0d72b6d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:42.051851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b977e1-1d1d-4ac2-8445-eb2cd0d72b6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:42.051851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b977e1-1d1d-4ac2-8445-eb2cd0d72b6d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:42.053175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:23:42.059021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a7a3-f3e8-4ee6-96bb-85a210eddc7d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:42.059490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a7a3-f3e8-4ee6-96bb-85a210eddc7d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:42.059612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c501a7a3-f3e8-4ee6-96bb-85a210eddc7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:42.081253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c501a7a3-f3e8-4ee6-96bb-85a210eddc7d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:23:42.082513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c501a7a3-f3e8-4ee6-96bb-85a210eddc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:42.090762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:23:42.170408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b977e1-1d1d-4ac2-8445-eb2cd0d72b6d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:23:42.172252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f9b977e1-1d1d-4ac2-8445-eb2cd0d72b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:42.257211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b977e1-1d1d-4ac2-8445-eb2cd0d72b6d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:23:42.260977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b46b8f-c9f4-43d4-bdf4-cb40c95f983a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:23:42.261607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b46b8f-c9f4-43d4-bdf4-cb40c95f983a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:42.261920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b46b8f-c9f4-43d4-bdf4-cb40c95f983a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:42.262194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b46b8f-c9f4-43d4-bdf4-cb40c95f983a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:42.272717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b46b8f-c9f4-43d4-bdf4-cb40c95f983a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:23:42.274243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b46b8f-c9f4-43d4-bdf4-cb40c95f983a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:23:42.284914 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 767/1530] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:23:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:23:44.314668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90fa45-f15f-4249-9e20-a6145188c6d0 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:44.315274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90fa45-f15f-4249-9e20-a6145188c6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:44.315383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae90fa45-f15f-4249-9e20-a6145188c6d0 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:44.317130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:23:44.323270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc762e9d-4ade-4997-abb6-39df1fb8a0f3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:44.323750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc762e9d-4ade-4997-abb6-39df1fb8a0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:44.324326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc762e9d-4ade-4997-abb6-39df1fb8a0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:44.344602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-dc762e9d-4ade-4997-abb6-39df1fb8a0f3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:23:44.345858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc762e9d-4ade-4997-abb6-39df1fb8a0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:44.354639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:23:44.391565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90fa45-f15f-4249-9e20-a6145188c6d0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:23:44.393622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ae90fa45-f15f-4249-9e20-a6145188c6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:44.407949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae90fa45-f15f-4249-9e20-a6145188c6d0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:23:44.411721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd83f4ff-827a-4641-802d-8a80ae228131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:23:44.412352 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83f4ff-827a-4641-802d-8a80ae228131 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:44.412669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83f4ff-827a-4641-802d-8a80ae228131 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:44.412938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd83f4ff-827a-4641-802d-8a80ae228131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:44.422689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83f4ff-827a-4641-802d-8a80ae228131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:23:44.424182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd83f4ff-827a-4641-802d-8a80ae228131 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:23:44.435501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 765/1533] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:23:48.096993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e1d5b-4448-4e05-83ab-f07adba41c01 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:48.097575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e1d5b-4448-4e05-83ab-f07adba41c01 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:48.097575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e1d5b-4448-4e05-83ab-f07adba41c01 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:48.098939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:23:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:23:48.104612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688ff35-3c77-490c-9b83-b21a9b9fe972 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:48.105074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688ff35-3c77-490c-9b83-b21a9b9fe972 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:48.105178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688ff35-3c77-490c-9b83-b21a9b9fe972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:48.113248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:23:48.126472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d688ff35-3c77-490c-9b83-b21a9b9fe972 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:23:48.127675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d688ff35-3c77-490c-9b83-b21a9b9fe972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:48.136215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:23:48.441877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e1d5b-4448-4e05-83ab-f07adba41c01 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:23:48.443949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-638e1d5b-4448-4e05-83ab-f07adba41c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:48.511916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:23:48.526454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e1d5b-4448-4e05-83ab-f07adba41c01 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:23:48.530252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a635d5-b647-44b9-a4eb-c4b90901b772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:23:48.530885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a635d5-b647-44b9-a4eb-c4b90901b772 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:48.531167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a635d5-b647-44b9-a4eb-c4b90901b772 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:48.531442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a635d5-b647-44b9-a4eb-c4b90901b772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:48.541089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a635d5-b647-44b9-a4eb-c4b90901b772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:23:48.542547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a635d5-b647-44b9-a4eb-c4b90901b772 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:23:48.552475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 770/1536] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:23:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:23:50.566075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12863ed-e309-42c2-85bb-6025648bae3c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:50.566641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12863ed-e309-42c2-85bb-6025648bae3c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:50.566641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12863ed-e309-42c2-85bb-6025648bae3c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:50.568074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 26 07:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:23:50.573449 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b005f-e4f9-449b-95d4-523c1b57dc95 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:50.573950 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b005f-e4f9-449b-95d4-523c1b57dc95 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:50.574058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b005f-e4f9-449b-95d4-523c1b57dc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:50.595095 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cb7b005f-e4f9-449b-95d4-523c1b57dc95 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:23:50.596192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cb7b005f-e4f9-449b-95d4-523c1b57dc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:50.605484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 26 07:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:23:50.907179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12863ed-e309-42c2-85bb-6025648bae3c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:23:50.909108 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d12863ed-e309-42c2-85bb-6025648bae3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:23:50.923203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12863ed-e309-42c2-85bb-6025648bae3c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:23:50.927055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ea227-3764-4a5d-88f0-e866e7b67b17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:23:50.927651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ea227-3764-4a5d-88f0-e866e7b67b17 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:23:50.927951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ea227-3764-4a5d-88f0-e866e7b67b17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:23:50.928219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ea227-3764-4a5d-88f0-e866e7b67b17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:23:50.938372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ea227-3764-4a5d-88f0-e866e7b67b17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:23:50.939842 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ea227-3764-4a5d-88f0-e866e7b67b17 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:23:50.949656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 768/1539] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:24:00.589927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=11 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:24:00.945313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=8 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:19.289535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550142a-789b-4ac3-b211-7308d4fc5c73 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.291451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550142a-789b-4ac3-b211-7308d4fc5c73 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.291451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550142a-789b-4ac3-b211-7308d4fc5c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.318478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-3550142a-789b-4ac3-b211-7308d4fc5c73 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:19.320162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3550142a-789b-4ac3-b211-7308d4fc5c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:19.332600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 772/1540] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:19.345700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a386-5faa-4798-9397-b3b1e8f49cc4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.346349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a386-5faa-4798-9397-b3b1e8f49cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.346349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d6a386-5faa-4798-9397-b3b1e8f49cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.373415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f9d6a386-5faa-4798-9397-b3b1e8f49cc4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:19.379071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f9d6a386-5faa-4798-9397-b3b1e8f49cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:19.388392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 769/1541] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:19.393607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550142a-789b-4ac3-b211-7308d4fc5c73 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:19.409791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550142a-789b-4ac3-b211-7308d4fc5c73 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:19.416714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809965c8-7a9f-46fa-a334-8057ae1dcecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:19.417884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809965c8-7a9f-46fa-a334-8057ae1dcecc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.418403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809965c8-7a9f-46fa-a334-8057ae1dcecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.418904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809965c8-7a9f-46fa-a334-8057ae1dcecc admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.419895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809965c8-7a9f-46fa-a334-8057ae1dcecc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:19.420813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809965c8-7a9f-46fa-a334-8057ae1dcecc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:19.422802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809965c8-7a9f-46fa-a334-8057ae1dcecc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:19.442488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 773/1542] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:19.451735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6a386-5faa-4798-9397-b3b1e8f49cc4 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:19.466706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d6a386-5faa-4798-9397-b3b1e8f49cc4 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:19.478826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d9bce-0d9b-42dc-9a5f-4a8bde3c2d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:19.480604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d9bce-0d9b-42dc-9a5f-4a8bde3c2d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.480604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d9bce-0d9b-42dc-9a5f-4a8bde3c2d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.480604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8d9bce-0d9b-42dc-9a5f-4a8bde3c2d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.480887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d9bce-0d9b-42dc-9a5f-4a8bde3c2d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:19.482726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8d9bce-0d9b-42dc-9a5f-4a8bde3c2d1f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:19.545281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 770/1543] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:19 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:19.550358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809965c8-7a9f-46fa-a334-8057ae1dcecc admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:19.563031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809965c8-7a9f-46fa-a334-8057ae1dcecc admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:19.644549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe8116-9026-481c-85c3-352e5bcc8b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:19.645627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8116-9026-481c-85c3-352e5bcc8b93 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.646184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8116-9026-481c-85c3-352e5bcc8b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.646691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe8116-9026-481c-85c3-352e5bcc8b93 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.647574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe8116-9026-481c-85c3-352e5bcc8b93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:19.649764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe8116-9026-481c-85c3-352e5bcc8b93 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:19.680802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 774/1544] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:19 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:19.685468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d9bce-0d9b-42dc-9a5f-4a8bde3c2d1f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:19.704458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8d9bce-0d9b-42dc-9a5f-4a8bde3c2d1f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:19.720768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5b5be-8197-4855-bb49-373e9ec03fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:19.722037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5b5be-8197-4855-bb49-373e9ec03fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.722663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5b5be-8197-4855-bb49-373e9ec03fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.723131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa5b5be-8197-4855-bb49-373e9ec03fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.724326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5b5be-8197-4855-bb49-373e9ec03fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:19.726178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa5b5be-8197-4855-bb49-373e9ec03fd4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:19.745675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ca502-a7bc-4671-b821-7075f142ce48 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.746377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ca502-a7bc-4671-b821-7075f142ce48 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.746940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 771/1545] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:19.747330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ca502-a7bc-4671-b821-7075f142ce48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.754309 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5b5be-8197-4855-bb49-373e9ec03fd4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:19.772522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa5b5be-8197-4855-bb49-373e9ec03fd4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:19.780921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a16ca502-a7bc-4671-b821-7075f142ce48 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:19.780921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a16ca502-a7bc-4671-b821-7075f142ce48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:19.784071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676a874-21ab-4e16-9c72-0ab712257c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:19.784728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676a874-21ab-4e16-9c72-0ab712257c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.785011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676a874-21ab-4e16-9c72-0ab712257c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.785285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676a874-21ab-4e16-9c72-0ab712257c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.786023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676a874-21ab-4e16-9c72-0ab712257c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:19.787640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676a874-21ab-4e16-9c72-0ab712257c08 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:19.803302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 775/1546] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:25:19.804527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44243fde-db7d-4cd3-8648-246ea0af52eb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.804923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44243fde-db7d-4cd3-8648-246ea0af52eb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.805219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44243fde-db7d-4cd3-8648-246ea0af52eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.805626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 772/1547] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:25:19.809974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676a874-21ab-4e16-9c72-0ab712257c08 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:19.828344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676a874-21ab-4e16-9c72-0ab712257c08 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:19.845193 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087173e3-effa-4c02-9bac-82371becde62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:19.845526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087173e3-effa-4c02-9bac-82371becde62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.845649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087173e3-effa-4c02-9bac-82371becde62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.845912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087173e3-effa-4c02-9bac-82371becde62 admin admin] PATH_INFO: `/v3/projects/0874292e3f8f4db98adc2b9672f840c4/users/08c4ec6a688c4608aa75aaf046933eec/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.846043 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-44243fde-db7d-4cd3-8648-246ea0af52eb None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:19.847243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-44243fde-db7d-4cd3-8648-246ea0af52eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:19.875784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087173e3-effa-4c02-9bac-82371becde62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0874292e3f8f4db98adc2b9672f840c4, user_id=08c4ec6a688c4608aa75aaf046933eec, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:19.876023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 776/1548] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:19.877226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc9247-2e14-4923-b40b-3ed41021e9c7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.877517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc9247-2e14-4923-b40b-3ed41021e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.877827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc9247-2e14-4923-b40b-3ed41021e9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.878615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087173e3-effa-4c02-9bac-82371becde62 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:19.914723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 773/1549] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:19 2022] PUT /identity/v3/projects/0874292e3f8f4db98adc2b9672f840c4/users/08c4ec6a688c4608aa75aaf046933eec/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:19.916871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-30bc9247-2e14-4923-b40b-3ed41021e9c7 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:19.918153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-30bc9247-2e14-4923-b40b-3ed41021e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:19.919213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3227ab4-55af-4d2a-8a00-ba96e859035d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:19.919467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3227ab4-55af-4d2a-8a00-ba96e859035d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:19.919750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3227ab4-55af-4d2a-8a00-ba96e859035d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:19.947950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f3227ab4-55af-4d2a-8a00-ba96e859035d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:19.949195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f3227ab4-55af-4d2a-8a00-ba96e859035d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.008125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 777/1550] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.010570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e250d8-73fa-4062-8064-b59170a6e00b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.011059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e250d8-73fa-4062-8064-b59170a6e00b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.011275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e250d8-73fa-4062-8064-b59170a6e00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.017690 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 774/1551] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.019077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06a265-0eed-440b-8701-5254aa9b19b4 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.019387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06a265-0eed-440b-8701-5254aa9b19b4 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.019693 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06a265-0eed-440b-8701-5254aa9b19b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.034345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f0e250d8-73fa-4062-8064-b59170a6e00b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.035563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f0e250d8-73fa-4062-8064-b59170a6e00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.038571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7e06a265-0eed-440b-8701-5254aa9b19b4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.039742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e06a265-0eed-440b-8701-5254aa9b19b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.045859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 778/1552] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.045859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b9e79-6e47-4f2a-8dc4-3482603e1007 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.045859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b9e79-6e47-4f2a-8dc4-3482603e1007 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.045859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b9e79-6e47-4f2a-8dc4-3482603e1007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.053990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 775/1553] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.055534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c05540-86fb-410f-b1ad-e25f32b61672 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.055940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c05540-86fb-410f-b1ad-e25f32b61672 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.056219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c05540-86fb-410f-b1ad-e25f32b61672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.066275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-441b9e79-6e47-4f2a-8dc4-3482603e1007 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.067515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-441b9e79-6e47-4f2a-8dc4-3482603e1007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.075537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 779/1554] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.076100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b2c05540-86fb-410f-b1ad-e25f32b61672 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.076841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2f92f-6f8d-4f2d-8538-24d3fbb2a627 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.077130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2f92f-6f8d-4f2d-8538-24d3fbb2a627 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.077318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b2c05540-86fb-410f-b1ad-e25f32b61672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.077426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2f92f-6f8d-4f2d-8538-24d3fbb2a627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.089302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 776/1555] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.090701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38057c-65eb-49c4-947f-44c721cd39d1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.090985 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38057c-65eb-49c4-947f-44c721cd39d1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.091262 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38057c-65eb-49c4-947f-44c721cd39d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.099189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4cd2f92f-6f8d-4f2d-8538-24d3fbb2a627 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.100386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4cd2f92f-6f8d-4f2d-8538-24d3fbb2a627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.108271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 780/1556] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.110777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cb38057c-65eb-49c4-947f-44c721cd39d1 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.111822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2f92f-6f8d-4f2d-8538-24d3fbb2a627 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.111970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb38057c-65eb-49c4-947f-44c721cd39d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.120132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 777/1557] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.120985 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38057c-65eb-49c4-947f-44c721cd39d1 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.122284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2f92f-6f8d-4f2d-8538-24d3fbb2a627 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.127521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3fead-c657-415a-9f0c-ed005b9bf165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.128104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3fead-c657-415a-9f0c-ed005b9bf165 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.128392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3fead-c657-415a-9f0c-ed005b9bf165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.128665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3fead-c657-415a-9f0c-ed005b9bf165 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.129377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3fead-c657-415a-9f0c-ed005b9bf165 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:20.129679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3fead-c657-415a-9f0c-ed005b9bf165 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.130917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38057c-65eb-49c4-947f-44c721cd39d1 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.131324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3fead-c657-415a-9f0c-ed005b9bf165 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.135444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981e1ff-3d6b-4eb3-aad0-58dc1c760c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.136048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981e1ff-3d6b-4eb3-aad0-58dc1c760c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.136327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981e1ff-3d6b-4eb3-aad0-58dc1c760c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.136598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981e1ff-3d6b-4eb3-aad0-58dc1c760c0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.137289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981e1ff-3d6b-4eb3-aad0-58dc1c760c0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:20.137590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981e1ff-3d6b-4eb3-aad0-58dc1c760c0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.138676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-8981e1ff-3d6b-4eb3-aad0-58dc1c760c0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:25:20.138676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 26 07:25:20.138676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: Jul 26 07:25:20.139706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981e1ff-3d6b-4eb3-aad0-58dc1c760c0e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.144209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 781/1558] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:20.145422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3fead-c657-415a-9f0c-ed005b9bf165 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.159253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 778/1559] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:20.159253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895644b8-3d83-400e-b24c-94d60f33c039 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.159253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895644b8-3d83-400e-b24c-94d60f33c039 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.159253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895644b8-3d83-400e-b24c-94d60f33c039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.174249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-895644b8-3d83-400e-b24c-94d60f33c039 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.175514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-895644b8-3d83-400e-b24c-94d60f33c039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.182747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:20.183557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 779/1560] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.184704 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873052e-8987-4c55-9c7f-692cdb127377 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.185044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873052e-8987-4c55-9c7f-692cdb127377 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.185312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f873052e-8987-4c55-9c7f-692cdb127377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.205635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f873052e-8987-4c55-9c7f-692cdb127377 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.206833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f873052e-8987-4c55-9c7f-692cdb127377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.215062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 780/1561] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.215223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3fead-c657-415a-9f0c-ed005b9bf165 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.216179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4f6ac-ad88-4650-99e2-ae5846bfac16 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.216460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4f6ac-ad88-4650-99e2-ae5846bfac16 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.216727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf4f6ac-ad88-4650-99e2-ae5846bfac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.220243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81ae17-2661-4ff8-bb42-2758e5df1e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.220894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81ae17-2661-4ff8-bb42-2758e5df1e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.221147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81ae17-2661-4ff8-bb42-2758e5df1e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.221412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81ae17-2661-4ff8-bb42-2758e5df1e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.231166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81ae17-2661-4ff8-bb42-2758e5df1e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.232605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81ae17-2661-4ff8-bb42-2758e5df1e81 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.238026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7bf4f6ac-ad88-4650-99e2-ae5846bfac16 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.239219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7bf4f6ac-ad88-4650-99e2-ae5846bfac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.242519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 782/1562] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:20.247105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c89c6-a0f8-4f21-8023-4cad34ef4b6c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.247395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c89c6-a0f8-4f21-8023-4cad34ef4b6c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.247668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c89c6-a0f8-4f21-8023-4cad34ef4b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.248288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 781/1563] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.249481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65427b3-162f-4f23-be0e-723194d6f0dd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.249764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65427b3-162f-4f23-be0e-723194d6f0dd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.250078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65427b3-162f-4f23-be0e-723194d6f0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.271057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f65427b3-162f-4f23-be0e-723194d6f0dd None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.271872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bb8c89c6-a0f8-4f21-8023-4cad34ef4b6c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:20.272299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f65427b3-162f-4f23-be0e-723194d6f0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.273042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bb8c89c6-a0f8-4f21-8023-4cad34ef4b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:20.285332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 782/1564] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.285332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65427b3-162f-4f23-be0e-723194d6f0dd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.285634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 783/1565] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:20.285634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c89c6-a0f8-4f21-8023-4cad34ef4b6c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.294173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c89c6-a0f8-4f21-8023-4cad34ef4b6c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.295648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65427b3-162f-4f23-be0e-723194d6f0dd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.299404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0173e-5d89-422a-a37e-7af16cae87b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.300040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0173e-5d89-422a-a37e-7af16cae87b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.300318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0173e-5d89-422a-a37e-7af16cae87b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.300515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a0705-7c27-46ba-8a31-3598cf8233e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.300640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0173e-5d89-422a-a37e-7af16cae87b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.301139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0705-7c27-46ba-8a31-3598cf8233e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.301246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0173e-5d89-422a-a37e-7af16cae87b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.301416 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0705-7c27-46ba-8a31-3598cf8233e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.301692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a0705-7c27-46ba-8a31-3598cf8233e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.302340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-c9d0173e-5d89-422a-a37e-7af16cae87b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:25:20.302340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 26 07:25:20.302340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: Jul 26 07:25:20.302752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a0705-7c27-46ba-8a31-3598cf8233e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.303485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0173e-5d89-422a-a37e-7af16cae87b4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.304441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a0705-7c27-46ba-8a31-3598cf8233e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.324009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:20.334899 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 784/1566] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:20.336485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0173e-5d89-422a-a37e-7af16cae87b4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.342097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 783/1567] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:20.342839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a0705-7c27-46ba-8a31-3598cf8233e8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.410763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0173e-5d89-422a-a37e-7af16cae87b4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.427289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a0705-7c27-46ba-8a31-3598cf8233e8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.430676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094386e-f2d5-40e2-a52c-a117b0bd88d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.431661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094386e-f2d5-40e2-a52c-a117b0bd88d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.431996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094386e-f2d5-40e2-a52c-a117b0bd88d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.432398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094386e-f2d5-40e2-a52c-a117b0bd88d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.433119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094386e-f2d5-40e2-a52c-a117b0bd88d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:20.433471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094386e-f2d5-40e2-a52c-a117b0bd88d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.435222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094386e-f2d5-40e2-a52c-a117b0bd88d1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.460296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 784/1568] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:20.462877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094386e-f2d5-40e2-a52c-a117b0bd88d1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.480383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094386e-f2d5-40e2-a52c-a117b0bd88d1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.488468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3b0ca-66eb-424d-b30d-901a16b5ca53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.489266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3b0ca-66eb-424d-b30d-901a16b5ca53 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.489616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3b0ca-66eb-424d-b30d-901a16b5ca53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.490382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b3b0ca-66eb-424d-b30d-901a16b5ca53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.491205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3b0ca-66eb-424d-b30d-901a16b5ca53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:20.491599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3b0ca-66eb-424d-b30d-901a16b5ca53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.493769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b3b0ca-66eb-424d-b30d-901a16b5ca53 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.500423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea468349-e401-47ab-8266-6a4f26e035b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.501169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea468349-e401-47ab-8266-6a4f26e035b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.501658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea468349-e401-47ab-8266-6a4f26e035b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.502247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea468349-e401-47ab-8266-6a4f26e035b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.503030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea468349-e401-47ab-8266-6a4f26e035b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:20.503402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea468349-e401-47ab-8266-6a4f26e035b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.505191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea468349-e401-47ab-8266-6a4f26e035b1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.508484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 785/1569] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:20.509483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3b0ca-66eb-424d-b30d-901a16b5ca53 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.522406 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b3b0ca-66eb-424d-b30d-901a16b5ca53 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.525848 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 785/1570] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:20.526420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea468349-e401-47ab-8266-6a4f26e035b1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.528491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af3024-e96d-46b3-a060-79b43af25fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.529302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af3024-e96d-46b3-a060-79b43af25fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.529666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af3024-e96d-46b3-a060-79b43af25fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.530253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af3024-e96d-46b3-a060-79b43af25fe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.530722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af3024-e96d-46b3-a060-79b43af25fe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:20.531624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af3024-e96d-46b3-a060-79b43af25fe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.533957 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af3024-e96d-46b3-a060-79b43af25fe8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.539916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea468349-e401-47ab-8266-6a4f26e035b1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.546412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d349f05-f115-4b42-a052-c27ce20456c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.547046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d349f05-f115-4b42-a052-c27ce20456c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.547321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d349f05-f115-4b42-a052-c27ce20456c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.547592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d349f05-f115-4b42-a052-c27ce20456c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.548371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d349f05-f115-4b42-a052-c27ce20456c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:20.548679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d349f05-f115-4b42-a052-c27ce20456c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.550315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d349f05-f115-4b42-a052-c27ce20456c4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.554402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 786/1571] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:20.555245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af3024-e96d-46b3-a060-79b43af25fe8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.571947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 786/1572] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:20.571947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d349f05-f115-4b42-a052-c27ce20456c4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.572288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af3024-e96d-46b3-a060-79b43af25fe8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.576417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cfdea-d9c4-4c66-b2fd-5fb13e56f1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.577128 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cfdea-d9c4-4c66-b2fd-5fb13e56f1cb admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.577413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cfdea-d9c4-4c66-b2fd-5fb13e56f1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.577803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652cfdea-d9c4-4c66-b2fd-5fb13e56f1cb admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.578578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cfdea-d9c4-4c66-b2fd-5fb13e56f1cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.580063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-652cfdea-d9c4-4c66-b2fd-5fb13e56f1cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:25:20.580063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 26 07:25:20.580063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: Jul 26 07:25:20.581225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652cfdea-d9c4-4c66-b2fd-5fb13e56f1cb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.584079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d349f05-f115-4b42-a052-c27ce20456c4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.588596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4f1d2-e181-484c-949a-cabdcc4e703b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.589220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4f1d2-e181-484c-949a-cabdcc4e703b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.589502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4f1d2-e181-484c-949a-cabdcc4e703b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.589875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4f1d2-e181-484c-949a-cabdcc4e703b admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.590608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4f1d2-e181-484c-949a-cabdcc4e703b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.592747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4f1d2-e181-484c-949a-cabdcc4e703b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.610702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 787/1573] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/users => generated 293 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:20.611649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cfdea-d9c4-4c66-b2fd-5fb13e56f1cb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.621555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 787/1574] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/users => generated 291 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:20.622428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4f1d2-e181-484c-949a-cabdcc4e703b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.624535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652cfdea-d9c4-4c66-b2fd-5fb13e56f1cb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.629867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4f861-f5ba-4752-b448-d822517ec073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.630529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4f861-f5ba-4752-b448-d822517ec073 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.630800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4f861-f5ba-4752-b448-d822517ec073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.631100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4f861-f5ba-4752-b448-d822517ec073 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.631765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4f861-f5ba-4752-b448-d822517ec073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.633216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4f1d2-e181-484c-949a-cabdcc4e703b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.633755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4f861-f5ba-4752-b448-d822517ec073 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.638596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb05b4-c052-4e50-9ed1-9da8164d1ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.639289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb05b4-c052-4e50-9ed1-9da8164d1ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.639674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb05b4-c052-4e50-9ed1-9da8164d1ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.640024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb05b4-c052-4e50-9ed1-9da8164d1ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.640691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb05b4-c052-4e50-9ed1-9da8164d1ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.642819 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb05b4-c052-4e50-9ed1-9da8164d1ff8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.649836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:20.674197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 788/1575] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:20.675203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4f861-f5ba-4752-b448-d822517ec073 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.686892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 788/1576] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:20.688114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb05b4-c052-4e50-9ed1-9da8164d1ff8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.757581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4f861-f5ba-4752-b448-d822517ec073 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.762381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb05b4-c052-4e50-9ed1-9da8164d1ff8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.763614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51d3b4-1d1c-416a-b1c1-6501ee919487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.764257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51d3b4-1d1c-416a-b1c1-6501ee919487 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.764574 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51d3b4-1d1c-416a-b1c1-6501ee919487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.764844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a51d3b4-1d1c-416a-b1c1-6501ee919487 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.765550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51d3b4-1d1c-416a-b1c1-6501ee919487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.767253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddec51-aa32-4ce5-b5cd-c26b96b45ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.767532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a51d3b4-1d1c-416a-b1c1-6501ee919487 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.768004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddec51-aa32-4ce5-b5cd-c26b96b45ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.768378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddec51-aa32-4ce5-b5cd-c26b96b45ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.768704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ddec51-aa32-4ce5-b5cd-c26b96b45ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.769587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddec51-aa32-4ce5-b5cd-c26b96b45ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.771645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ddec51-aa32-4ce5-b5cd-c26b96b45ea6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.797493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 789/1577] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:20.798364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51d3b4-1d1c-416a-b1c1-6501ee919487 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.806537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 789/1578] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:20.807453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddec51-aa32-4ce5-b5cd-c26b96b45ea6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.816073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:20.870591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a51d3b4-1d1c-416a-b1c1-6501ee919487 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.875949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2828a-574d-45b2-8569-27694d705db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.876578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2828a-574d-45b2-8569-27694d705db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.876857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2828a-574d-45b2-8569-27694d705db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.877128 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2828a-574d-45b2-8569-27694d705db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.877817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2828a-574d-45b2-8569-27694d705db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.878912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ddec51-aa32-4ce5-b5cd-c26b96b45ea6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.879742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2828a-574d-45b2-8569-27694d705db7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.884130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175baca9-2b13-4f5b-8db3-f0b65b5d2aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.884770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175baca9-2b13-4f5b-8db3-f0b65b5d2aee admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.885104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175baca9-2b13-4f5b-8db3-f0b65b5d2aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.885393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175baca9-2b13-4f5b-8db3-f0b65b5d2aee admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.886137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175baca9-2b13-4f5b-8db3-f0b65b5d2aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.887811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175baca9-2b13-4f5b-8db3-f0b65b5d2aee admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.898351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 790/1579] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:20.899201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175baca9-2b13-4f5b-8db3-f0b65b5d2aee admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.909536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175baca9-2b13-4f5b-8db3-f0b65b5d2aee admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.914598 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d0ad2-d681-4e83-88a2-53744d7decb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:20.915084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 790/1580] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/projects => generated 393 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:20.915994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2828a-574d-45b2-8569-27694d705db7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.918371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d0ad2-d681-4e83-88a2-53744d7decb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:20.918667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d0ad2-d681-4e83-88a2-53744d7decb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:20.918952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161d0ad2-d681-4e83-88a2-53744d7decb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:20.919664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d0ad2-d681-4e83-88a2-53744d7decb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:20.923354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161d0ad2-d681-4e83-88a2-53744d7decb2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:20.934593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 791/1581] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:20.935461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d0ad2-d681-4e83-88a2-53744d7decb2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:20.947492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161d0ad2-d681-4e83-88a2-53744d7decb2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:20.992799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2828a-574d-45b2-8569-27694d705db7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.003496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86116396-6a87-4459-964b-9b50fe219d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.007334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86116396-6a87-4459-964b-9b50fe219d0d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.007334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86116396-6a87-4459-964b-9b50fe219d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.007334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86116396-6a87-4459-964b-9b50fe219d0d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.007334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86116396-6a87-4459-964b-9b50fe219d0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.007854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86116396-6a87-4459-964b-9b50fe219d0d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.030818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:21.034621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 791/1582] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/users => generated 290 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:21.036135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86116396-6a87-4459-964b-9b50fe219d0d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.040031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3e410-2e26-4c95-a404-2584912a518e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.041100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e410-2e26-4c95-a404-2584912a518e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.041883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e410-2e26-4c95-a404-2584912a518e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.042600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e410-2e26-4c95-a404-2584912a518e admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.043526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3e410-2e26-4c95-a404-2584912a518e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.047339 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f3e410-2e26-4c95-a404-2584912a518e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.049293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86116396-6a87-4459-964b-9b50fe219d0d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.055326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de6bf2-88b3-45ad-b76f-7ec8cf9f8c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.056164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de6bf2-88b3-45ad-b76f-7ec8cf9f8c21 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.056800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de6bf2-88b3-45ad-b76f-7ec8cf9f8c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.056925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de6bf2-88b3-45ad-b76f-7ec8cf9f8c21 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.057561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de6bf2-88b3-45ad-b76f-7ec8cf9f8c21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.059726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de6bf2-88b3-45ad-b76f-7ec8cf9f8c21 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.087054 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 792/1583] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:20 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:21.087054 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3e410-2e26-4c95-a404-2584912a518e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.090061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 792/1584] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:21 2022] POST /identity/v3/users => generated 293 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:21.092234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de6bf2-88b3-45ad-b76f-7ec8cf9f8c21 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.096057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3e410-2e26-4c95-a404-2584912a518e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.104131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bcb5b-e314-41f0-a516-0dc985c3a7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.104909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bcb5b-e314-41f0-a516-0dc985c3a7ce admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.105194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bcb5b-e314-41f0-a516-0dc985c3a7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.105309 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de6bf2-88b3-45ad-b76f-7ec8cf9f8c21 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.105470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60bcb5b-e314-41f0-a516-0dc985c3a7ce admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.106269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bcb5b-e314-41f0-a516-0dc985c3a7ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.108674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60bcb5b-e314-41f0-a516-0dc985c3a7ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.110515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8a0e5-cf84-4842-a650-736a3637d650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.111148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8a0e5-cf84-4842-a650-736a3637d650 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.111548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8a0e5-cf84-4842-a650-736a3637d650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.111828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be8a0e5-cf84-4842-a650-736a3637d650 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.112529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8a0e5-cf84-4842-a650-736a3637d650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.114318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be8a0e5-cf84-4842-a650-736a3637d650 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.130845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 793/1585] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.130845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8a0e5-cf84-4842-a650-736a3637d650 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.142569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 793/1586] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:21 2022] POST /identity/v3/users => generated 287 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:21.143479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bcb5b-e314-41f0-a516-0dc985c3a7ce admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.151422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be8a0e5-cf84-4842-a650-736a3637d650 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.154314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60bcb5b-e314-41f0-a516-0dc985c3a7ce admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.159217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c126e97-78b8-4e66-9d79-90028ab2cb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.159422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072adc8-f208-4fc5-8180-196acfc4ba6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.160155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072adc8-f208-4fc5-8180-196acfc4ba6b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.160434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072adc8-f208-4fc5-8180-196acfc4ba6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.160713 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072adc8-f208-4fc5-8180-196acfc4ba6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.161237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c126e97-78b8-4e66-9d79-90028ab2cb12 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.161415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072adc8-f208-4fc5-8180-196acfc4ba6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.162415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c126e97-78b8-4e66-9d79-90028ab2cb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.162881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c126e97-78b8-4e66-9d79-90028ab2cb12 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.163183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072adc8-f208-4fc5-8180-196acfc4ba6b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.164024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c126e97-78b8-4e66-9d79-90028ab2cb12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.167135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c126e97-78b8-4e66-9d79-90028ab2cb12 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.175593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 794/1587] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.176428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072adc8-f208-4fc5-8180-196acfc4ba6b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.188359 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072adc8-f208-4fc5-8180-196acfc4ba6b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.194230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3f28-fb86-45b7-a410-e883a594f973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.195003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3f28-fb86-45b7-a410-e883a594f973 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.195286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3f28-fb86-45b7-a410-e883a594f973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.195565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da3f28-fb86-45b7-a410-e883a594f973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.196483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da3f28-fb86-45b7-a410-e883a594f973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.198178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20da3f28-fb86-45b7-a410-e883a594f973 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.200037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 794/1588] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:21 2022] POST /identity/v3/users => generated 297 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:21.201791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c126e97-78b8-4e66-9d79-90028ab2cb12 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.211843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 795/1589] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.212559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3f28-fb86-45b7-a410-e883a594f973 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.217662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c126e97-78b8-4e66-9d79-90028ab2cb12 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.225265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f7d2c-3e81-483b-9122-890133b5e5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.225988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da3f28-fb86-45b7-a410-e883a594f973 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.226489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f7d2c-3e81-483b-9122-890133b5e5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.226822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f7d2c-3e81-483b-9122-890133b5e5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.227106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f7d2c-3e81-483b-9122-890133b5e5f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.228025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f7d2c-3e81-483b-9122-890133b5e5f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.229944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717f7d2c-3e81-483b-9122-890133b5e5f8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.233854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff01c-ecdc-4cff-8a49-19126f648e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.234501 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff01c-ecdc-4cff-8a49-19126f648e88 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.234776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff01c-ecdc-4cff-8a49-19126f648e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.235100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ff01c-ecdc-4cff-8a49-19126f648e88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.235813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ff01c-ecdc-4cff-8a49-19126f648e88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.237582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ff01c-ecdc-4cff-8a49-19126f648e88 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.248196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 795/1590] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.249188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f7d2c-3e81-483b-9122-890133b5e5f8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.255520 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 796/1591] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.256685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff01c-ecdc-4cff-8a49-19126f648e88 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.260336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717f7d2c-3e81-483b-9122-890133b5e5f8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.267173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde003e-5fa0-4cca-8df5-28a767030dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.267923 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde003e-5fa0-4cca-8df5-28a767030dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.268213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde003e-5fa0-4cca-8df5-28a767030dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.268640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde003e-5fa0-4cca-8df5-28a767030dde admin admin] PATH_INFO: `/v3/projects/4fa0c5574ef9497ebe29658f8164764c/users/57029844165f4d83892176fe82594685/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.269351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ff01c-ecdc-4cff-8a49-19126f648e88 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.274897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f441977-3976-4c79-a99f-6850b299f6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.275522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f441977-3976-4c79-a99f-6850b299f6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.275802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f441977-3976-4c79-a99f-6850b299f6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.276075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f441977-3976-4c79-a99f-6850b299f6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.277078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f441977-3976-4c79-a99f-6850b299f6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.279774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f441977-3976-4c79-a99f-6850b299f6c7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.287622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde003e-5fa0-4cca-8df5-28a767030dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa0c5574ef9497ebe29658f8164764c, user_id=57029844165f4d83892176fe82594685, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.291255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde003e-5fa0-4cca-8df5-28a767030dde admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.298507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 797/1592] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.298507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f441977-3976-4c79-a99f-6850b299f6c7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.310257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f441977-3976-4c79-a99f-6850b299f6c7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.318742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 796/1593] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:21 2022] PUT /identity/v3/projects/4fa0c5574ef9497ebe29658f8164764c/users/57029844165f4d83892176fe82594685/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:21.320100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde003e-5fa0-4cca-8df5-28a767030dde admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.326355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:21.377398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5f00d-28cb-4dc7-8a13-c48f057787d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.378567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5f00d-28cb-4dc7-8a13-c48f057787d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.378870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5f00d-28cb-4dc7-8a13-c48f057787d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.379153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5f00d-28cb-4dc7-8a13-c48f057787d3 admin admin] PATH_INFO: `/v3/projects/df0856fb51f6430bb262c1c6aa6625f5/users/cfb4f20954d34751a96beb40be86300a/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.397681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af5f00d-28cb-4dc7-8a13-c48f057787d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0856fb51f6430bb262c1c6aa6625f5, user_id=cfb4f20954d34751a96beb40be86300a, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.399256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-2af5f00d-28cb-4dc7-8a13-c48f057787d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:25:21.399256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 26 07:25:21.399256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: Jul 26 07:25:21.401324 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde003e-5fa0-4cca-8df5-28a767030dde admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.401717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af5f00d-28cb-4dc7-8a13-c48f057787d3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.407564 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fb165-9566-4adb-b9b6-97a1439e7a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.408547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fb165-9566-4adb-b9b6-97a1439e7a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.408916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fb165-9566-4adb-b9b6-97a1439e7a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.409233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fb165-9566-4adb-b9b6-97a1439e7a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.410097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fb165-9566-4adb-b9b6-97a1439e7a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.411852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46fb165-9566-4adb-b9b6-97a1439e7a59 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.422356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 798/1594] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:21 2022] PUT /identity/v3/projects/df0856fb51f6430bb262c1c6aa6625f5/users/cfb4f20954d34751a96beb40be86300a/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:21.424339 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5f00d-28cb-4dc7-8a13-c48f057787d3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.431866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 797/1595] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.432767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fb165-9566-4adb-b9b6-97a1439e7a59 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.448078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fb165-9566-4adb-b9b6-97a1439e7a59 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.509578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af5f00d-28cb-4dc7-8a13-c48f057787d3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.516993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f6fe0-aae9-4d0b-a184-7a8a0402c3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.517886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f6fe0-aae9-4d0b-a184-7a8a0402c3be admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.518163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f6fe0-aae9-4d0b-a184-7a8a0402c3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.518443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f6fe0-aae9-4d0b-a184-7a8a0402c3be admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.520456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f6fe0-aae9-4d0b-a184-7a8a0402c3be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.521474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f6fe0-aae9-4d0b-a184-7a8a0402c3be admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.532008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebe321-b832-44b6-ab0f-bff2b9908f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.532658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebe321-b832-44b6-ab0f-bff2b9908f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.532937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebe321-b832-44b6-ab0f-bff2b9908f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.533212 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebe321-b832-44b6-ab0f-bff2b9908f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.534139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebe321-b832-44b6-ab0f-bff2b9908f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.536124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebe321-b832-44b6-ab0f-bff2b9908f68 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.538432 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 799/1596] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.539413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f6fe0-aae9-4d0b-a184-7a8a0402c3be admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.546720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:21.554507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 798/1597] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.554705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f6fe0-aae9-4d0b-a184-7a8a0402c3be admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.555272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebe321-b832-44b6-ab0f-bff2b9908f68 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.562139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf132d7-a513-48b6-a56d-bcbd47d51c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.563220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf132d7-a513-48b6-a56d-bcbd47d51c23 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.563703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf132d7-a513-48b6-a56d-bcbd47d51c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.564174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf132d7-a513-48b6-a56d-bcbd47d51c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.565092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf132d7-a513-48b6-a56d-bcbd47d51c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.567061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebe321-b832-44b6-ab0f-bff2b9908f68 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.568256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf132d7-a513-48b6-a56d-bcbd47d51c23 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.572581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad2087-9534-41b3-96aa-ef0f54bffb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.573346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad2087-9534-41b3-96aa-ef0f54bffb69 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.573642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad2087-9534-41b3-96aa-ef0f54bffb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.574121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad2087-9534-41b3-96aa-ef0f54bffb69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.574845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad2087-9534-41b3-96aa-ef0f54bffb69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.576485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad2087-9534-41b3-96aa-ef0f54bffb69 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.587474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 800/1598] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.589158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14e919-fa21-493e-8e06-489e9b54489a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.589694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14e919-fa21-493e-8e06-489e9b54489a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.590241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f14e919-fa21-493e-8e06-489e9b54489a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.597807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 799/1599] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.598771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad2087-9534-41b3-96aa-ef0f54bffb69 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.611328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad2087-9534-41b3-96aa-ef0f54bffb69 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.616257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8f14e919-fa21-493e-8e06-489e9b54489a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:21.618431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f14e919-fa21-493e-8e06-489e9b54489a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:21.618659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa8ebd-cd04-4e3b-aefb-61eb738e1274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.619306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa8ebd-cd04-4e3b-aefb-61eb738e1274 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.619733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa8ebd-cd04-4e3b-aefb-61eb738e1274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.620010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa8ebd-cd04-4e3b-aefb-61eb738e1274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.620722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa8ebd-cd04-4e3b-aefb-61eb738e1274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.622502 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa8ebd-cd04-4e3b-aefb-61eb738e1274 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.630599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 801/1600] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:21.632444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc85fa-fe5f-46d0-8475-17ab2e610b57 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.632937 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc85fa-fe5f-46d0-8475-17ab2e610b57 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.633633 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cc85fa-fe5f-46d0-8475-17ab2e610b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.647017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 800/1601] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:21.648894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa8ebd-cd04-4e3b-aefb-61eb738e1274 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.665246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa8ebd-cd04-4e3b-aefb-61eb738e1274 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.671565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0925a-0e5d-43fa-ba91-8c1a1fca4ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.672303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0925a-0e5d-43fa-ba91-8c1a1fca4ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.672587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0925a-0e5d-43fa-ba91-8c1a1fca4ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.673501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0925a-0e5d-43fa-ba91-8c1a1fca4ecf admin admin] PATH_INFO: `/v3/projects/500111b9c9ca4d8893e64c488668a491/users/8fdc64e58cb041f98d9ea4b8c8cd9e60/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.677038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d7cc85fa-fe5f-46d0-8475-17ab2e610b57 None None] MFA Rules not processed for user `cfb4f20954d34751a96beb40be86300a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:21.678548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d7cc85fa-fe5f-46d0-8475-17ab2e610b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:21.689226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0925a-0e5d-43fa-ba91-8c1a1fca4ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=500111b9c9ca4d8893e64c488668a491, user_id=8fdc64e58cb041f98d9ea4b8c8cd9e60, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.694457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0925a-0e5d-43fa-ba91-8c1a1fca4ecf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.717855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 801/1602] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:21 2022] PUT /identity/v3/projects/500111b9c9ca4d8893e64c488668a491/users/8fdc64e58cb041f98d9ea4b8c8cd9e60/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:21.718880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0925a-0e5d-43fa-ba91-8c1a1fca4ecf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.775327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 802/1603] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:21.777110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc85fa-fe5f-46d0-8475-17ab2e610b57 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.784759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:21.794937 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cc85fa-fe5f-46d0-8475-17ab2e610b57 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.802041 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb2d1d-f1a7-4b55-b689-a94ef73c85e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.803250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb2d1d-f1a7-4b55-b689-a94ef73c85e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.803539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb2d1d-f1a7-4b55-b689-a94ef73c85e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.805015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb2d1d-f1a7-4b55-b689-a94ef73c85e0 admin admin] PATH_INFO: `/v3/projects/9c39dec587e24c84b3c9702e177a2dbd/users/1a58299139db4f02beca97cd8feed5f9/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.809507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0925a-0e5d-43fa-ba91-8c1a1fca4ecf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.818488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb937f-d9be-4c09-b65b-73e5e76dc283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.819667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb937f-d9be-4c09-b65b-73e5e76dc283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.820293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb937f-d9be-4c09-b65b-73e5e76dc283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.820955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb937f-d9be-4c09-b65b-73e5e76dc283 admin admin] PATH_INFO: `/v3/projects/cd9ccb653b7d4c7ebc362319c284c1c5/users/a0d5ec89a5064c43b4eb31ae77565d15/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.825796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb2d1d-f1a7-4b55-b689-a94ef73c85e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c39dec587e24c84b3c9702e177a2dbd, user_id=1a58299139db4f02beca97cd8feed5f9, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.829370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb2d1d-f1a7-4b55-b689-a94ef73c85e0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.844686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb937f-d9be-4c09-b65b-73e5e76dc283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd9ccb653b7d4c7ebc362319c284c1c5, user_id=a0d5ec89a5064c43b4eb31ae77565d15, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:21.848313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb937f-d9be-4c09-b65b-73e5e76dc283 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:21.863004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 803/1604] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:21 2022] PUT /identity/v3/projects/9c39dec587e24c84b3c9702e177a2dbd/users/1a58299139db4f02beca97cd8feed5f9/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:21.868038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 802/1605] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:21 2022] PUT /identity/v3/projects/cd9ccb653b7d4c7ebc362319c284c1c5/users/a0d5ec89a5064c43b4eb31ae77565d15/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:21.869092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb937f-d9be-4c09-b65b-73e5e76dc283 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.875889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb2d1d-f1a7-4b55-b689-a94ef73c85e0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:21.963625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb937f-d9be-4c09-b65b-73e5e76dc283 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:21.971489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a90380-0ebb-4f19-a4ad-dade60cec2a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:21.972273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a90380-0ebb-4f19-a4ad-dade60cec2a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:21.972555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a90380-0ebb-4f19-a4ad-dade60cec2a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:21.972989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a90380-0ebb-4f19-a4ad-dade60cec2a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:21.973968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb2d1d-f1a7-4b55-b689-a94ef73c85e0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:22.049335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a90380-0ebb-4f19-a4ad-dade60cec2a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:22.050907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a90380-0ebb-4f19-a4ad-dade60cec2a7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:22.053648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b09760-f402-4aa8-8eba-4268a2a19630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:22.054233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b09760-f402-4aa8-8eba-4268a2a19630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.054582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b09760-f402-4aa8-8eba-4268a2a19630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.055561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b09760-f402-4aa8-8eba-4268a2a19630 admin admin] PATH_INFO: `/v3/projects/8d86f999900147c0a2fa7d75d9ce90fb/users/618965bf5e3046ecace7edb01420bd17/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.057802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:22.077383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 803/1606] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:22.077594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b09760-f402-4aa8-8eba-4268a2a19630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d86f999900147c0a2fa7d75d9ce90fb, user_id=618965bf5e3046ecace7edb01420bd17, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:22.078890 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a90380-0ebb-4f19-a4ad-dade60cec2a7 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:22.081256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b09760-f402-4aa8-8eba-4268a2a19630 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:22.105855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 804/1607] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:21 2022] PUT /identity/v3/projects/8d86f999900147c0a2fa7d75d9ce90fb/users/618965bf5e3046ecace7edb01420bd17/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:22.109199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b63cf4-7675-477a-9a86-4eeead4d1ae0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.112061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b63cf4-7675-477a-9a86-4eeead4d1ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.113114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b63cf4-7675-477a-9a86-4eeead4d1ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.119887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a90380-0ebb-4f19-a4ad-dade60cec2a7 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:22.192007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-39b63cf4-7675-477a-9a86-4eeead4d1ae0 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:22.192007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-39b63cf4-7675-477a-9a86-4eeead4d1ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:22.247577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a728eb-91d3-4ebf-b8ef-1f94f4916e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:22.248963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a728eb-91d3-4ebf-b8ef-1f94f4916e76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.249416 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a728eb-91d3-4ebf-b8ef-1f94f4916e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.250218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a728eb-91d3-4ebf-b8ef-1f94f4916e76 admin admin] PATH_INFO: `/v3/projects/02e545f0c2784698a35cd8dcb1b02395/users/886151888fc84ccf890d609ba62da9e1/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.276019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a728eb-91d3-4ebf-b8ef-1f94f4916e76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e545f0c2784698a35cd8dcb1b02395, user_id=886151888fc84ccf890d609ba62da9e1, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:22.280732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a728eb-91d3-4ebf-b8ef-1f94f4916e76 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:22.309132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 805/1608] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:22.311510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b63cf4-7675-477a-9a86-4eeead4d1ae0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:22.312515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 804/1609] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:22 2022] PUT /identity/v3/projects/02e545f0c2784698a35cd8dcb1b02395/users/886151888fc84ccf890d609ba62da9e1/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:22.314847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2ece5-d672-4f62-9c87-8b9766ce0573 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.315213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2ece5-d672-4f62-9c87-8b9766ce0573 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.315491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2ece5-d672-4f62-9c87-8b9766ce0573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.324071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b63cf4-7675-477a-9a86-4eeead4d1ae0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:22.328199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:22.370317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-90f2ece5-d672-4f62-9c87-8b9766ce0573 None None] MFA Rules not processed for user `1a58299139db4f02beca97cd8feed5f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:22.371745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-90f2ece5-d672-4f62-9c87-8b9766ce0573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:22.442565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c97403-a91a-43d6-a06b-3412f768b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:22.443851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97403-a91a-43d6-a06b-3412f768b461 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.444300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97403-a91a-43d6-a06b-3412f768b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.444779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c97403-a91a-43d6-a06b-3412f768b461 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.446031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c97403-a91a-43d6-a06b-3412f768b461 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:22.451354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c97403-a91a-43d6-a06b-3412f768b461 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:22.482275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 805/1610] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:22.483811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eba322-a7db-4240-81e1-615335200b0f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.484536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eba322-a7db-4240-81e1-615335200b0f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.484536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eba322-a7db-4240-81e1-615335200b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.505177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 806/1611] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:22 2022] POST /identity/v3/projects => generated 377 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:22.508860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf5265-77a4-4dee-8ede-7b8c372a8240 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.509367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf5265-77a4-4dee-8ede-7b8c372a8240 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.510042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf5265-77a4-4dee-8ede-7b8c372a8240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.531691 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-58eba322-a7db-4240-81e1-615335200b0f None None] MFA Rules not processed for user `a0d5ec89a5064c43b4eb31ae77565d15`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:22.555116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-58eba322-a7db-4240-81e1-615335200b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:22.561222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-87bf5265-77a4-4dee-8ede-7b8c372a8240 None None] MFA Rules not processed for user `618965bf5e3046ecace7edb01420bd17`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:22.565534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-87bf5265-77a4-4dee-8ede-7b8c372a8240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:22.658617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 806/1612] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:22.660046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f904452-5f30-4ae2-b770-8ccef9592a9f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.660553 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f904452-5f30-4ae2-b770-8ccef9592a9f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.660665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f904452-5f30-4ae2-b770-8ccef9592a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.724281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9f904452-5f30-4ae2-b770-8ccef9592a9f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:22.728777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:22.732744 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9f904452-5f30-4ae2-b770-8ccef9592a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:22.735001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 807/1613] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:22.736247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf5265-77a4-4dee-8ede-7b8c372a8240 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:22.836695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf5265-77a4-4dee-8ede-7b8c372a8240 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:22.843426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ae70e-9630-4b6b-b5b3-654726f1b03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:22.844553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ae70e-9630-4b6b-b5b3-654726f1b03d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.845653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ae70e-9630-4b6b-b5b3-654726f1b03d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.846101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ae70e-9630-4b6b-b5b3-654726f1b03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.854355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 807/1614] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:22.855260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f904452-5f30-4ae2-b770-8ccef9592a9f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:22.867569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f904452-5f30-4ae2-b770-8ccef9592a9f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:22.875614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097c8f9-f25d-4c70-bd2d-96df131b5125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:22.876283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097c8f9-f25d-4c70-bd2d-96df131b5125 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.876679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097c8f9-f25d-4c70-bd2d-96df131b5125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.877168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097c8f9-f25d-4c70-bd2d-96df131b5125 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.878050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097c8f9-f25d-4c70-bd2d-96df131b5125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:22.884816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ae70e-9630-4b6b-b5b3-654726f1b03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:22.886900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097c8f9-f25d-4c70-bd2d-96df131b5125 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:22.887872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ae70e-9630-4b6b-b5b3-654726f1b03d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:22.903522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 808/1615] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:22.905391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312a85e-4eeb-4f92-8ce0-642275625132 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.905894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312a85e-4eeb-4f92-8ce0-642275625132 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.906383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312a85e-4eeb-4f92-8ce0-642275625132 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.916096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 809/1616] 10.222.0.39 () {54 vars in 962 bytes} [Tue Jul 26 07:25:22 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:25:22.920833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d3261-1a18-41e4-9089-76a9feac3211 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.921575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d3261-1a18-41e4-9089-76a9feac3211 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.922060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d3261-1a18-41e4-9089-76a9feac3211 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.925143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 810/1617] 10.222.0.39 () {54 vars in 962 bytes} [Tue Jul 26 07:25:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:25:22.926172 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d3261-1a18-41e4-9089-76a9feac3211 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:22.931212 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 808/1618] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:22 2022] POST /identity/v3/users => generated 295 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:22.935620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66d682-0a47-4d04-acaf-c50ab276b140 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.935975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66d682-0a47-4d04-acaf-c50ab276b140 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.936402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a66d682-0a47-4d04-acaf-c50ab276b140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.950036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d3261-1a18-41e4-9089-76a9feac3211 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:22.960340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7223ee-babe-4f2c-81f2-7a127dbc6370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:22.961248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7223ee-babe-4f2c-81f2-7a127dbc6370 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:22.962743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7223ee-babe-4f2c-81f2-7a127dbc6370 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:22.963350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:22.964277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7223ee-babe-4f2c-81f2-7a127dbc6370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:22.991867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7223ee-babe-4f2c-81f2-7a127dbc6370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:22.996226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7223ee-babe-4f2c-81f2-7a127dbc6370 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:23.022374 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 811/1619] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:23.025069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd4f0e-3c3f-4f79-85ee-241f61b1fb9a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.025700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd4f0e-3c3f-4f79-85ee-241f61b1fb9a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.027328 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bd4f0e-3c3f-4f79-85ee-241f61b1fb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:23.028646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3a66d682-0a47-4d04-acaf-c50ab276b140 None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:23.032260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a66d682-0a47-4d04-acaf-c50ab276b140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:23.076413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-36bd4f0e-3c3f-4f79-85ee-241f61b1fb9a None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:23.079686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-36bd4f0e-3c3f-4f79-85ee-241f61b1fb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:23.198802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 809/1620] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Jul 26 07:25:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:23.199853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66d682-0a47-4d04-acaf-c50ab276b140 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:23.213731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a66d682-0a47-4d04-acaf-c50ab276b140 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:23.221507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf4e36-befc-4110-aa9f-7825b893937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:23.222292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4e36-befc-4110-aa9f-7825b893937c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.222594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4e36-befc-4110-aa9f-7825b893937c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.222883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf4e36-befc-4110-aa9f-7825b893937c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:23.223876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf4e36-befc-4110-aa9f-7825b893937c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:23.225872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf4e36-befc-4110-aa9f-7825b893937c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:23.247234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 810/1621] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:23.248229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf4e36-befc-4110-aa9f-7825b893937c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:23.283473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 812/1622] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Jul 26 07:25:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:23.284550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd4f0e-3c3f-4f79-85ee-241f61b1fb9a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:23.290351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf4e36-befc-4110-aa9f-7825b893937c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:23.297200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee34a4-4f25-4d6e-90c6-1ccc555ba24d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:23.297937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee34a4-4f25-4d6e-90c6-1ccc555ba24d service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.298224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee34a4-4f25-4d6e-90c6-1ccc555ba24d service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.298535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee34a4-4f25-4d6e-90c6-1ccc555ba24d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:23.319375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bd4f0e-3c3f-4f79-85ee-241f61b1fb9a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:23.335691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1d94b-b41c-4018-a148-e3c082c99f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:23.338204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1d94b-b41c-4018-a148-e3c082c99f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.339626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1d94b-b41c-4018-a148-e3c082c99f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.340105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1d94b-b41c-4018-a148-e3c082c99f92 admin admin] PATH_INFO: `/v3/projects/8a8e01f61251484c970e66ba5c4ec0b3/users/e8254bf5fabb43fabef2931437b41971/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:23.374021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee34a4-4f25-4d6e-90c6-1ccc555ba24d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:23.376736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee34a4-4f25-4d6e-90c6-1ccc555ba24d service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:23.389037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1d94b-b41c-4018-a148-e3c082c99f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8e01f61251484c970e66ba5c4ec0b3, user_id=e8254bf5fabb43fabef2931437b41971, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:23.389830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 811/1623] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:23.390856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee34a4-4f25-4d6e-90c6-1ccc555ba24d service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:23.392621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1d94b-b41c-4018-a148-e3c082c99f92 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:23.405319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee34a4-4f25-4d6e-90c6-1ccc555ba24d service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:23.414117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017d5f5-ec9a-427e-87cc-5574090cb8a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:23.415042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017d5f5-ec9a-427e-87cc-5574090cb8a6 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.415428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017d5f5-ec9a-427e-87cc-5574090cb8a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.416007 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017d5f5-ec9a-427e-87cc-5574090cb8a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:23.434250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 813/1624] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:23 2022] PUT /identity/v3/projects/8a8e01f61251484c970e66ba5c4ec0b3/users/e8254bf5fabb43fabef2931437b41971/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:23.439829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1d94b-b41c-4018-a148-e3c082c99f92 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:23.442122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017d5f5-ec9a-427e-87cc-5574090cb8a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:23.444102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017d5f5-ec9a-427e-87cc-5574090cb8a6 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:23.459808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 812/1625] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:23.538551 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:23.545240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1d94b-b41c-4018-a148-e3c082c99f92 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:23.554844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78a717-3b8c-44bc-83b8-57813f11b19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:23.555908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a717-3b8c-44bc-83b8-57813f11b19c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.556555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a717-3b8c-44bc-83b8-57813f11b19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.557294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78a717-3b8c-44bc-83b8-57813f11b19c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:23.558262 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78a717-3b8c-44bc-83b8-57813f11b19c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:23.560524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78a717-3b8c-44bc-83b8-57813f11b19c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:23.581859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 814/1626] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:23.583710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017d5f5-ec9a-427e-87cc-5574090cb8a6 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:23.596477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017d5f5-ec9a-427e-87cc-5574090cb8a6 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:23.605598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d2f557-646a-4feb-9ccc-fe066426bf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:23.606647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2f557-646a-4feb-9ccc-fe066426bf59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.607292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2f557-646a-4feb-9ccc-fe066426bf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.612224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d2f557-646a-4feb-9ccc-fe066426bf59 admin admin] PATH_INFO: `/v3/projects/8a8e01f61251484c970e66ba5c4ec0b3/users/e8254bf5fabb43fabef2931437b41971/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:23.616776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2f557-646a-4feb-9ccc-fe066426bf59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8e01f61251484c970e66ba5c4ec0b3, user_id=e8254bf5fabb43fabef2931437b41971, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:23.621531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d2f557-646a-4feb-9ccc-fe066426bf59 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:23.656591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 813/1627] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:23 2022] PUT /identity/v3/projects/8a8e01f61251484c970e66ba5c4ec0b3/users/e8254bf5fabb43fabef2931437b41971/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:23.667441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8d781-f4dd-46d2-84bb-635678afc2a7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.667733 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8d781-f4dd-46d2-84bb-635678afc2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.668021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8d781-f4dd-46d2-84bb-635678afc2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:23.727678 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d1d8d781-f4dd-46d2-84bb-635678afc2a7 None None] MFA Rules not processed for user `e8254bf5fabb43fabef2931437b41971`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:23.728935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1d8d781-f4dd-46d2-84bb-635678afc2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:23.753402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b0a60-59d6-4bf6-8f93-99e999de0316 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.753689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b0a60-59d6-4bf6-8f93-99e999de0316 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.754001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b0a60-59d6-4bf6-8f93-99e999de0316 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:23.755380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 814/1628] 10.222.0.39 () {54 vars in 912 bytes} [Tue Jul 26 07:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:25:23.773361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef296-990e-4b16-ad9a-e3e45f9ff315 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.773646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef296-990e-4b16-ad9a-e3e45f9ff315 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.773961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1ef296-990e-4b16-ad9a-e3e45f9ff315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:23.810065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9b1ef296-990e-4b16-ad9a-e3e45f9ff315 None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:23.811619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9b1ef296-990e-4b16-ad9a-e3e45f9ff315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:23.916253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 815/1629] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:23.931073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8d781-f4dd-46d2-84bb-635678afc2a7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:23.941458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:23.984358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 815/1630] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:25:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:23.988739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a9f90-b7f7-43a3-ad1b-5502a22f882d None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:23.989113 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a9f90-b7f7-43a3-ad1b-5502a22f882d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:23.989468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a9f90-b7f7-43a3-ad1b-5502a22f882d None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:24.005949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 816/1631] 10.222.0.39 () {54 vars in 912 bytes} [Tue Jul 26 07:25:23 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:25:24.008217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a9f90-b7f7-43a3-ad1b-5502a22f882d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:24.057003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8d781-f4dd-46d2-84bb-635678afc2a7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:24.064402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3e238-35c5-445f-8f84-719012078a7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:24.065814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3e238-35c5-445f-8f84-719012078a7e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:24.066096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3e238-35c5-445f-8f84-719012078a7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:24.067630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3e238-35c5-445f-8f84-719012078a7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:24.082875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3e238-35c5-445f-8f84-719012078a7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:24.084836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3e238-35c5-445f-8f84-719012078a7e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:24.107811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 816/1632] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:24.109408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372c04c-fd40-451f-b06b-357eaf309ce5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:24.109706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372c04c-fd40-451f-b06b-357eaf309ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:24.110030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372c04c-fd40-451f-b06b-357eaf309ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:24.135393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d372c04c-fd40-451f-b06b-357eaf309ce5 None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:24.136897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d372c04c-fd40-451f-b06b-357eaf309ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:24.157807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a9f90-b7f7-43a3-ad1b-5502a22f882d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:24.159966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 817/1633] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:25:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:24.165612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998d404-ebd9-4cda-9f4f-bab1cd52761e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:24.166628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998d404-ebd9-4cda-9f4f-bab1cd52761e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:24.167374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998d404-ebd9-4cda-9f4f-bab1cd52761e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:24.167701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1998d404-ebd9-4cda-9f4f-bab1cd52761e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:24.185347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998d404-ebd9-4cda-9f4f-bab1cd52761e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:24.187267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1998d404-ebd9-4cda-9f4f-bab1cd52761e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:24.204389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 817/1634] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:24.220945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372c04c-fd40-451f-b06b-357eaf309ce5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:24.235684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372c04c-fd40-451f-b06b-357eaf309ce5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:24.241933 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f3e53-e687-42cc-a5fe-ba72a3ffb099 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:24.242714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f3e53-e687-42cc-a5fe-ba72a3ffb099 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:24.243088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f3e53-e687-42cc-a5fe-ba72a3ffb099 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:24.243760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f3e53-e687-42cc-a5fe-ba72a3ffb099 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:24.257587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f3e53-e687-42cc-a5fe-ba72a3ffb099 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:24.259606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f3e53-e687-42cc-a5fe-ba72a3ffb099 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:24.276297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 818/1635] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:25.263622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998d404-ebd9-4cda-9f4f-bab1cd52761e service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:25.294994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f3e53-e687-42cc-a5fe-ba72a3ffb099 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:25.306921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1998d404-ebd9-4cda-9f4f-bab1cd52761e service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:25.338569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f3e53-e687-42cc-a5fe-ba72a3ffb099 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:25.344476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c442e33-8884-451f-888d-49d04a1450eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:25.345446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442e33-8884-451f-888d-49d04a1450eb service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:25.345729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442e33-8884-451f-888d-49d04a1450eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:25.346036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c442e33-8884-451f-888d-49d04a1450eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:25.363994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe199ee-7270-404c-9a80-bcaf05456b03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:25.364638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe199ee-7270-404c-9a80-bcaf05456b03 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:25.364920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe199ee-7270-404c-9a80-bcaf05456b03 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:25.365194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe199ee-7270-404c-9a80-bcaf05456b03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:25.552739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe199ee-7270-404c-9a80-bcaf05456b03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:25.555007 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe199ee-7270-404c-9a80-bcaf05456b03 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:25.570811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c442e33-8884-451f-888d-49d04a1450eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:25.588218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c442e33-8884-451f-888d-49d04a1450eb service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:25.593535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:25.599023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 818/1636] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:25.615139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 819/1637] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:25.938182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe199ee-7270-404c-9a80-bcaf05456b03 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:26.059819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe199ee-7270-404c-9a80-bcaf05456b03 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:26.065712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:26.066583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:26.066877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:26.067203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] PATH_INFO: `/v3/users/618965bf5e3046ecace7edb01420bd17` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:26.070970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] RBAC: Authorizing `identity:delete_user(user_id=618965bf5e3046ecace7edb01420bd17)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:26.075527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:26.109704 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:26.167280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '618965bf5e3046ecace7edb01420bd17'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:26.168250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '618965bf5e3046ecace7edb01420bd17'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:26.193568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '618965bf5e3046ecace7edb01420bd17'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:26.207088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '618965bf5e3046ecace7edb01420bd17'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:26.221393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '618965bf5e3046ecace7edb01420bd17'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:26.241589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ee559-c248-4a23-9f59-65bda2d0e11d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:26.242184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ee559-c248-4a23-9f59-65bda2d0e11d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:26.242671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ee559-c248-4a23-9f59-65bda2d0e11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:26.246346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 819/1638] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:25 2022] DELETE /identity/v3/users/618965bf5e3046ecace7edb01420bd17 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:26.294520 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0a3ee559-c248-4a23-9f59-65bda2d0e11d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:26.296260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a3ee559-c248-4a23-9f59-65bda2d0e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:26.441676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 820/1639] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:26.461588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:26.463354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:26.656756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604eade4-18b2-4c9c-a58e-b9f95b1fc2db admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:26.671670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908beebc-e72b-4c14-9d99-84db77350e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:26.672341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908beebc-e72b-4c14-9d99-84db77350e10 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:26.672628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908beebc-e72b-4c14-9d99-84db77350e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:26.672905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908beebc-e72b-4c14-9d99-84db77350e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:26.691042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908beebc-e72b-4c14-9d99-84db77350e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:26.692530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908beebc-e72b-4c14-9d99-84db77350e10 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:26.731360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 820/1640] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:28.381012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ee559-c248-4a23-9f59-65bda2d0e11d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:28.384726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0a3ee559-c248-4a23-9f59-65bda2d0e11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:28.416976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ee559-c248-4a23-9f59-65bda2d0e11d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:28.442233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:28.442907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:28.443614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:28.444108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] PATH_INFO: `/v3/projects/8d86f999900147c0a2fa7d75d9ce90fb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:28.445837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d86f999900147c0a2fa7d75d9ce90fb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:28.450199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:28.536440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d86f999900147c0a2fa7d75d9ce90fb', 'request_id': 'req-990274fd-4f73-4cff-a20a-ac050a4f7a99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:28.551417 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d86f999900147c0a2fa7d75d9ce90fb', 'request_id': 'req-990274fd-4f73-4cff-a20a-ac050a4f7a99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:28.570148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d86f999900147c0a2fa7d75d9ce90fb', 'request_id': 'req-990274fd-4f73-4cff-a20a-ac050a4f7a99'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:28.597608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] The token cache is being invalidate because project 8d86f999900147c0a2fa7d75d9ce90fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:25:28.598302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:28.602279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-990274fd-4f73-4cff-a20a-ac050a4f7a99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:28.608341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 821/1641] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:25:28 2022] DELETE /identity/v3/projects/8d86f999900147c0a2fa7d75d9ce90fb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:28.610354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9178a-d7e2-45f0-b35f-1ff4896555de None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:28.611636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9178a-d7e2-45f0-b35f-1ff4896555de None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:28.612060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9178a-d7e2-45f0-b35f-1ff4896555de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:28.691085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-52c9178a-d7e2-45f0-b35f-1ff4896555de None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:28.692777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-52c9178a-d7e2-45f0-b35f-1ff4896555de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:28.900849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 821/1642] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:28.909258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f233ad-0864-4734-933a-394dba4fb1e2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:28.909902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f233ad-0864-4734-933a-394dba4fb1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:28.910396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f233ad-0864-4734-933a-394dba4fb1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:28.937813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-94f233ad-0864-4734-933a-394dba4fb1e2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:28.939551 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94f233ad-0864-4734-933a-394dba4fb1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:28.952944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 822/1643] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:28.954692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9178a-d7e2-45f0-b35f-1ff4896555de None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:28.976851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9178a-d7e2-45f0-b35f-1ff4896555de None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:28.987374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21283fc3-18cb-4986-a18b-fd657543dea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:28.988111 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21283fc3-18cb-4986-a18b-fd657543dea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:28.988373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21283fc3-18cb-4986-a18b-fd657543dea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:28.988559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21283fc3-18cb-4986-a18b-fd657543dea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:28.989580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21283fc3-18cb-4986-a18b-fd657543dea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:28.990557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21283fc3-18cb-4986-a18b-fd657543dea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:28.996350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21283fc3-18cb-4986-a18b-fd657543dea2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:29.033549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 822/1644] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:25:29.038048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f233ad-0864-4734-933a-394dba4fb1e2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:29.082699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f233ad-0864-4734-933a-394dba4fb1e2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:29.093879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbc33b-f195-45d0-b86c-f260a27e6ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:29.095336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc33b-f195-45d0-b86c-f260a27e6ae2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:29.099631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc33b-f195-45d0-b86c-f260a27e6ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:29.100236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cbc33b-f195-45d0-b86c-f260a27e6ae2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:29.104231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbc33b-f195-45d0-b86c-f260a27e6ae2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:29.106619 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cbc33b-f195-45d0-b86c-f260a27e6ae2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:29.176882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 823/1645] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:29 2022] POST /identity/v3/projects => generated 393 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:29.181783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21283fc3-18cb-4986-a18b-fd657543dea2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:29.197055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21283fc3-18cb-4986-a18b-fd657543dea2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:29.260742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:29.349167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46afd41-d6b6-47f8-87f1-b36581bdc4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:29.350146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46afd41-d6b6-47f8-87f1-b36581bdc4db admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:29.351494 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46afd41-d6b6-47f8-87f1-b36581bdc4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:29.352105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46afd41-d6b6-47f8-87f1-b36581bdc4db admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:29.352955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46afd41-d6b6-47f8-87f1-b36581bdc4db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:29.355181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46afd41-d6b6-47f8-87f1-b36581bdc4db admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:29.401798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 823/1646] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:29 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:29.404205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbc33b-f195-45d0-b86c-f260a27e6ae2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:29.418569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cbc33b-f195-45d0-b86c-f260a27e6ae2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:29.426536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558743b6-e5b1-4df7-ac9f-88d2f23cb14d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:29.427219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558743b6-e5b1-4df7-ac9f-88d2f23cb14d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:29.427499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558743b6-e5b1-4df7-ac9f-88d2f23cb14d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:29.428004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558743b6-e5b1-4df7-ac9f-88d2f23cb14d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:29.428851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558743b6-e5b1-4df7-ac9f-88d2f23cb14d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:29.430814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558743b6-e5b1-4df7-ac9f-88d2f23cb14d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:29.449290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 824/1647] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:29.454124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46afd41-d6b6-47f8-87f1-b36581bdc4db admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:29.472236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46afd41-d6b6-47f8-87f1-b36581bdc4db admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:29.486339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a5c06-7054-4924-b4fc-837ef10a18b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:29.487092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a5c06-7054-4924-b4fc-837ef10a18b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:29.487440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a5c06-7054-4924-b4fc-837ef10a18b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:29.487773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a5c06-7054-4924-b4fc-837ef10a18b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:29.488582 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a5c06-7054-4924-b4fc-837ef10a18b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:29.490352 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a5c06-7054-4924-b4fc-837ef10a18b6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:29.517690 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 824/1648] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:29.522015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558743b6-e5b1-4df7-ac9f-88d2f23cb14d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:29.542354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558743b6-e5b1-4df7-ac9f-88d2f23cb14d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:29.558240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c46fb0-33b1-4253-bafa-efd8ac6e11bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:29.560160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c46fb0-33b1-4253-bafa-efd8ac6e11bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:29.561009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c46fb0-33b1-4253-bafa-efd8ac6e11bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:29.563388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c46fb0-33b1-4253-bafa-efd8ac6e11bf admin admin] PATH_INFO: `/v3/projects/711f0e36d3a04f62bb410e01dd445053/users/9e87e1a57f8844fb8ad7b3fabd95a5ac/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:29.575878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6df41-9cfa-4f52-95d8-ef1feb30f37d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:29.576364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6df41-9cfa-4f52-95d8-ef1feb30f37d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:29.576766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6df41-9cfa-4f52-95d8-ef1feb30f37d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:29.598657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c46fb0-33b1-4253-bafa-efd8ac6e11bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=711f0e36d3a04f62bb410e01dd445053, user_id=9e87e1a57f8844fb8ad7b3fabd95a5ac, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:29.609331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c46fb0-33b1-4253-bafa-efd8ac6e11bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:29.638246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9bf6df41-9cfa-4f52-95d8-ef1feb30f37d None None] MFA Rules not processed for user `57029844165f4d83892176fe82594685`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:29.639772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9bf6df41-9cfa-4f52-95d8-ef1feb30f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:29.655895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 825/1649] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:29 2022] PUT /identity/v3/projects/711f0e36d3a04f62bb410e01dd445053/users/9e87e1a57f8844fb8ad7b3fabd95a5ac/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:29.659070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795da24-826a-44f6-84d7-042ebd6b50e9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:29.659367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795da24-826a-44f6-84d7-042ebd6b50e9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:29.660297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795da24-826a-44f6-84d7-042ebd6b50e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:29.799184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f795da24-826a-44f6-84d7-042ebd6b50e9 None None] MFA Rules not processed for user `9e87e1a57f8844fb8ad7b3fabd95a5ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:29.805518 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f795da24-826a-44f6-84d7-042ebd6b50e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:29.828010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:29.906318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 825/1650] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:25:29.945199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6df41-9cfa-4f52-95d8-ef1feb30f37d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:29.947068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9bf6df41-9cfa-4f52-95d8-ef1feb30f37d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:30.018526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 826/1651] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:25:30.021952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795da24-826a-44f6-84d7-042ebd6b50e9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:30.091078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6df41-9cfa-4f52-95d8-ef1feb30f37d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:30.107498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7af22-8e3b-402d-bbb7-21ce1dba6e14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:30.108241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7af22-8e3b-402d-bbb7-21ce1dba6e14 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:30.108527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7af22-8e3b-402d-bbb7-21ce1dba6e14 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:30.109107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c7af22-8e3b-402d-bbb7-21ce1dba6e14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:30.124300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7af22-8e3b-402d-bbb7-21ce1dba6e14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:30.126233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c7af22-8e3b-402d-bbb7-21ce1dba6e14 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:30.143707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795da24-826a-44f6-84d7-042ebd6b50e9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:30.154908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23b597-07c3-4fac-8dbb-33e631412051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:30.155506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 826/1652] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:25:30.156590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23b597-07c3-4fac-8dbb-33e631412051 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:30.156882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23b597-07c3-4fac-8dbb-33e631412051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:30.157302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23b597-07c3-4fac-8dbb-33e631412051 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:30.158074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23b597-07c3-4fac-8dbb-33e631412051 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:30.160514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23b597-07c3-4fac-8dbb-33e631412051 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:30.211388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 827/1653] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:30 2022] POST /identity/v3/projects => generated 391 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:30.211973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7af22-8e3b-402d-bbb7-21ce1dba6e14 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:30.230965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c7af22-8e3b-402d-bbb7-21ce1dba6e14 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:30.261775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:30.319484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c0fd-840b-4e38-8830-352772a51dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:30.320450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c0fd-840b-4e38-8830-352772a51dc0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:30.320913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c0fd-840b-4e38-8830-352772a51dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:30.321661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338c0fd-840b-4e38-8830-352772a51dc0 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:30.322501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338c0fd-840b-4e38-8830-352772a51dc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:30.325114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338c0fd-840b-4e38-8830-352772a51dc0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:30.390445 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 827/1654] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:30 2022] POST /identity/v3/users => generated 302 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:30.392296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23b597-07c3-4fac-8dbb-33e631412051 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:30.419649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23b597-07c3-4fac-8dbb-33e631412051 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:30.430845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c9d7-3d3a-453d-9a6b-ff9d0aae5fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:30.432833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c9d7-3d3a-453d-9a6b-ff9d0aae5fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:30.433660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c9d7-3d3a-453d-9a6b-ff9d0aae5fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:30.433660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170c9d7-3d3a-453d-9a6b-ff9d0aae5fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:30.435273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170c9d7-3d3a-453d-9a6b-ff9d0aae5fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:30.443040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170c9d7-3d3a-453d-9a6b-ff9d0aae5fc4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:30.471357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 828/1655] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:30.473275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c0fd-840b-4e38-8830-352772a51dc0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:30.495615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338c0fd-840b-4e38-8830-352772a51dc0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:30.501274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec302bb-9894-450d-9044-fe7cbdb1d173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:30.502301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec302bb-9894-450d-9044-fe7cbdb1d173 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:30.502680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec302bb-9894-450d-9044-fe7cbdb1d173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:30.503088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec302bb-9894-450d-9044-fe7cbdb1d173 admin admin] PATH_INFO: `/v3/projects/e0ac581979c44d3eaa7fcfd9fb9382e4/users/57b3f112691e441884e31971fd67e0ab/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:30.524301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec302bb-9894-450d-9044-fe7cbdb1d173 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ac581979c44d3eaa7fcfd9fb9382e4, user_id=57b3f112691e441884e31971fd67e0ab, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:30.530088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec302bb-9894-450d-9044-fe7cbdb1d173 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:30.556624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c9d7-3d3a-453d-9a6b-ff9d0aae5fc4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:30.557179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 828/1656] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:30 2022] PUT /identity/v3/projects/e0ac581979c44d3eaa7fcfd9fb9382e4/users/57b3f112691e441884e31971fd67e0ab/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:30.572524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170c9d7-3d3a-453d-9a6b-ff9d0aae5fc4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:30.681123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58b3de-64ac-48cf-b543-cfc51fa2a188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:30.682091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58b3de-64ac-48cf-b543-cfc51fa2a188 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:30.682512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58b3de-64ac-48cf-b543-cfc51fa2a188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:30.683108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58b3de-64ac-48cf-b543-cfc51fa2a188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:30.683992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58b3de-64ac-48cf-b543-cfc51fa2a188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:30.685817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58b3de-64ac-48cf-b543-cfc51fa2a188 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:30.706378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 829/1657] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:30.714782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec302bb-9894-450d-9044-fe7cbdb1d173 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:30.765945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec302bb-9894-450d-9044-fe7cbdb1d173 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:30.783995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c72819-dd5d-4fc4-ab9a-0ef056354013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:30.785000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c72819-dd5d-4fc4-ab9a-0ef056354013 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:30.785635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c72819-dd5d-4fc4-ab9a-0ef056354013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:30.786078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c72819-dd5d-4fc4-ab9a-0ef056354013 admin admin] PATH_INFO: `/v3/projects/e0ac581979c44d3eaa7fcfd9fb9382e4/users/57b3f112691e441884e31971fd67e0ab/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:30.790071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c72819-dd5d-4fc4-ab9a-0ef056354013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0ac581979c44d3eaa7fcfd9fb9382e4, user_id=57b3f112691e441884e31971fd67e0ab, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:30.797213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c72819-dd5d-4fc4-ab9a-0ef056354013 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:30.863146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 829/1658] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:30 2022] PUT /identity/v3/projects/e0ac581979c44d3eaa7fcfd9fb9382e4/users/57b3f112691e441884e31971fd67e0ab/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:30.882286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf7edb-bc71-49d1-bb45-d1a31be44db1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:30.882553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf7edb-bc71-49d1-bb45-d1a31be44db1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:30.882822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf7edb-bc71-49d1-bb45-d1a31be44db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:30.964791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ebaf7edb-bc71-49d1-bb45-d1a31be44db1 None None] MFA Rules not processed for user `57b3f112691e441884e31971fd67e0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:30.966335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ebaf7edb-bc71-49d1-bb45-d1a31be44db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:30.980862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:31.091194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c72819-dd5d-4fc4-ab9a-0ef056354013 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:31.093165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-35c72819-dd5d-4fc4-ab9a-0ef056354013 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:31.190757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 830/1659] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:31.202858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf7edb-bc71-49d1-bb45-d1a31be44db1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:31.206518 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ebaf7edb-bc71-49d1-bb45-d1a31be44db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:31.276910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c72819-dd5d-4fc4-ab9a-0ef056354013 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:31.295421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:31.296357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:31.296641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:31.296942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] PATH_INFO: `/v3/users/a0d5ec89a5064c43b4eb31ae77565d15` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:31.302177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d5ec89a5064c43b4eb31ae77565d15)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:31.308689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:31.368362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf7edb-bc71-49d1-bb45-d1a31be44db1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:31.377353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a341573-2a8d-4f29-938d-be01b17d5bb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:31.378849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a341573-2a8d-4f29-938d-be01b17d5bb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:31.380601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a341573-2a8d-4f29-938d-be01b17d5bb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:31.383852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a341573-2a8d-4f29-938d-be01b17d5bb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:31.408897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a341573-2a8d-4f29-938d-be01b17d5bb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:31.408897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a341573-2a8d-4f29-938d-be01b17d5bb1 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:31.447641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 831/1660] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:25:31.475857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d5ec89a5064c43b4eb31ae77565d15'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:31.478466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d5ec89a5064c43b4eb31ae77565d15'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:31.513479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d5ec89a5064c43b4eb31ae77565d15'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:31.539468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d5ec89a5064c43b4eb31ae77565d15'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:31.543787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:INSERT=6,keystone:DELETE=9 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:31.562778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d5ec89a5064c43b4eb31ae77565d15'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:31.571132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 830/1661] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:31 2022] DELETE /identity/v3/users/a0d5ec89a5064c43b4eb31ae77565d15 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:31.577732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c19f7b-5408-4b8c-a6c6-3a20d8ddee3e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:31.578010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c19f7b-5408-4b8c-a6c6-3a20d8ddee3e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:31.578330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c19f7b-5408-4b8c-a6c6-3a20d8ddee3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:31.583370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:31.585330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:31.644839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-65c19f7b-5408-4b8c-a6c6-3a20d8ddee3e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:31.649777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65c19f7b-5408-4b8c-a6c6-3a20d8ddee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:31.833833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0ec40-b225-41fa-942f-29ecd45c81f6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:31.842405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826f66b-3315-4b3b-b962-dacdb2807053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:31.845409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826f66b-3315-4b3b-b962-dacdb2807053 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:31.849021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826f66b-3315-4b3b-b962-dacdb2807053 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:31.849471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826f66b-3315-4b3b-b962-dacdb2807053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:31.859579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5826f66b-3315-4b3b-b962-dacdb2807053 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:31.871456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 832/1662] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:31.910223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c19f7b-5408-4b8c-a6c6-3a20d8ddee3e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:31.911675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65c19f7b-5408-4b8c-a6c6-3a20d8ddee3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:32.031850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:32.035255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826f66b-3315-4b3b-b962-dacdb2807053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:32.036815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826f66b-3315-4b3b-b962-dacdb2807053 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:32.043518 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c19f7b-5408-4b8c-a6c6-3a20d8ddee3e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:32.059900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4853af-f603-4842-8f34-4db1b76cecd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:32.060459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4853af-f603-4842-8f34-4db1b76cecd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:32.060864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4853af-f603-4842-8f34-4db1b76cecd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:32.061274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4853af-f603-4842-8f34-4db1b76cecd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:32.071032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 831/1663] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:32.077704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4853af-f603-4842-8f34-4db1b76cecd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:32.081100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4853af-f603-4842-8f34-4db1b76cecd1 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:32.098424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 833/1664] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:33.523820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826f66b-3315-4b3b-b962-dacdb2807053 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:33.525312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5826f66b-3315-4b3b-b962-dacdb2807053 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:33.556749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826f66b-3315-4b3b-b962-dacdb2807053 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:33.574023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:33.574722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:33.575005 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:33.575279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] PATH_INFO: `/v3/projects/cd9ccb653b7d4c7ebc362319c284c1c5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:33.576496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd9ccb653b7d4c7ebc362319c284c1c5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:33.577603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: WARNING py.warnings [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:25:33.577603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: warnings.warn(msg) Jul 26 07:25:33.577603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: Jul 26 07:25:33.578877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:33.711417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd9ccb653b7d4c7ebc362319c284c1c5', 'request_id': 'req-d8cd66e5-b098-4a6f-b857-9ab294045f6e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:33.750054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd9ccb653b7d4c7ebc362319c284c1c5', 'request_id': 'req-d8cd66e5-b098-4a6f-b857-9ab294045f6e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:33.765476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd9ccb653b7d4c7ebc362319c284c1c5', 'request_id': 'req-d8cd66e5-b098-4a6f-b857-9ab294045f6e'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:33.802775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] The token cache is being invalidate because project cd9ccb653b7d4c7ebc362319c284c1c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:25:33.803278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:33.810038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d8cd66e5-b098-4a6f-b857-9ab294045f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:33.816382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 832/1665] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:25:33 2022] DELETE /identity/v3/projects/cd9ccb653b7d4c7ebc362319c284c1c5 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:33.823414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f696d8-e0b6-49dc-80af-3bc7b5759b47 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:33.824714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f696d8-e0b6-49dc-80af-3bc7b5759b47 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:33.825200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f696d8-e0b6-49dc-80af-3bc7b5759b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:33.919845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b6f696d8-e0b6-49dc-80af-3bc7b5759b47 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:33.922077 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b6f696d8-e0b6-49dc-80af-3bc7b5759b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:34.124605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 834/1666] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:34.132661 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fc968-3a42-4685-8eca-7a8d1eabf442 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:34.133117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fc968-3a42-4685-8eca-7a8d1eabf442 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:34.133512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fc968-3a42-4685-8eca-7a8d1eabf442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:34.164534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0c6fc968-3a42-4685-8eca-7a8d1eabf442 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:34.166063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0c6fc968-3a42-4685-8eca-7a8d1eabf442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:34.176272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 833/1667] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:34.180884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f696d8-e0b6-49dc-80af-3bc7b5759b47 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:34.204123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f696d8-e0b6-49dc-80af-3bc7b5759b47 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:34.214691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963917c-91f5-4aba-af77-58a161f20492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:34.215625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963917c-91f5-4aba-af77-58a161f20492 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:34.216111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963917c-91f5-4aba-af77-58a161f20492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:34.216632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2963917c-91f5-4aba-af77-58a161f20492 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:34.217839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963917c-91f5-4aba-af77-58a161f20492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:34.218722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963917c-91f5-4aba-af77-58a161f20492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:34.224366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2963917c-91f5-4aba-af77-58a161f20492 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:34.291378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 835/1668] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:34.296264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fc968-3a42-4685-8eca-7a8d1eabf442 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:34.315435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fc968-3a42-4685-8eca-7a8d1eabf442 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:34.338174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6918e-dde1-4aad-888c-8b981a4eba6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:34.340053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6918e-dde1-4aad-888c-8b981a4eba6a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:34.340486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6918e-dde1-4aad-888c-8b981a4eba6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:34.340772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6918e-dde1-4aad-888c-8b981a4eba6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:34.344018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6918e-dde1-4aad-888c-8b981a4eba6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:34.352291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a6918e-dde1-4aad-888c-8b981a4eba6a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:34.465104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963917c-91f5-4aba-af77-58a161f20492 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:34.471576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 834/1669] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:34 2022] POST /identity/v3/projects => generated 367 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:34.482633 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2963917c-91f5-4aba-af77-58a161f20492 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:34.549033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb799-36cd-482f-ba9d-647b3be3e9e5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:34.549520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb799-36cd-482f-ba9d-647b3be3e9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:34.549695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb799-36cd-482f-ba9d-647b3be3e9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:34.560565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:34.576970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-35edb799-36cd-482f-ba9d-647b3be3e9e5 None None] MFA Rules not processed for user `57029844165f4d83892176fe82594685`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:34.578682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-35edb799-36cd-482f-ba9d-647b3be3e9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:34.691822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef81ce-b794-43a1-b0c7-9e9ca93f2c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:34.692566 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef81ce-b794-43a1-b0c7-9e9ca93f2c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:34.692854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef81ce-b794-43a1-b0c7-9e9ca93f2c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:34.693131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef81ce-b794-43a1-b0c7-9e9ca93f2c25 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:34.696109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef81ce-b794-43a1-b0c7-9e9ca93f2c25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:34.702503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef81ce-b794-43a1-b0c7-9e9ca93f2c25 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:34.712182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 835/1670] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:34.748998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 836/1671] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:34 2022] POST /identity/v3/users => generated 284 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:25:34.752940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edb799-36cd-482f-ba9d-647b3be3e9e5 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:34.772912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edb799-36cd-482f-ba9d-647b3be3e9e5 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:34.791329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6150840-3281-45ef-97d1-9b5771baae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:34.791996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6150840-3281-45ef-97d1-9b5771baae52 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:34.792445 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6150840-3281-45ef-97d1-9b5771baae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:34.792855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6150840-3281-45ef-97d1-9b5771baae52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:34.793772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6150840-3281-45ef-97d1-9b5771baae52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:34.795677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6150840-3281-45ef-97d1-9b5771baae52 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:34.830647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 836/1672] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:34.834391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef81ce-b794-43a1-b0c7-9e9ca93f2c25 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:34.850036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef81ce-b794-43a1-b0c7-9e9ca93f2c25 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:34.862451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737690a-1d0f-4316-99f8-7ec8305ea94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:34.863120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737690a-1d0f-4316-99f8-7ec8305ea94d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:34.863402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737690a-1d0f-4316-99f8-7ec8305ea94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:34.864099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737690a-1d0f-4316-99f8-7ec8305ea94d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:34.864980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737690a-1d0f-4316-99f8-7ec8305ea94d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:34.870651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737690a-1d0f-4316-99f8-7ec8305ea94d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:34.913844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 837/1673] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:34.918145 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6150840-3281-45ef-97d1-9b5771baae52 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:34.941124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6150840-3281-45ef-97d1-9b5771baae52 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:34.968192 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fd315-d8c8-4f2b-947c-8a54646fff73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:34.968989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fd315-d8c8-4f2b-947c-8a54646fff73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:34.969280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fd315-d8c8-4f2b-947c-8a54646fff73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:34.969562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fd315-d8c8-4f2b-947c-8a54646fff73 admin admin] PATH_INFO: `/v3/projects/19b0687625c44767bbca6729ab4dcd76/users/72f981865d484de8877fdbbe300799bb/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:35.051546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fd315-d8c8-4f2b-947c-8a54646fff73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19b0687625c44767bbca6729ab4dcd76, user_id=72f981865d484de8877fdbbe300799bb, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:35.056867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72fd315-d8c8-4f2b-947c-8a54646fff73 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:35.092623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 837/1674] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:34 2022] PUT /identity/v3/projects/19b0687625c44767bbca6729ab4dcd76/users/72f981865d484de8877fdbbe300799bb/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:35.107584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166c777-bd5b-4e61-97b0-83c23dea8e51 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:35.109059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166c777-bd5b-4e61-97b0-83c23dea8e51 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:35.109498 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f166c777-bd5b-4e61-97b0-83c23dea8e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:35.176574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f166c777-bd5b-4e61-97b0-83c23dea8e51 None None] MFA Rules not processed for user `72f981865d484de8877fdbbe300799bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:35.177855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f166c777-bd5b-4e61-97b0-83c23dea8e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:35.281309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:35.323816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 838/1675] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:35.330166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fd315-d8c8-4f2b-947c-8a54646fff73 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:35.472304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fd315-d8c8-4f2b-947c-8a54646fff73 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:35.486836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535151a1-f1fb-4bbf-aad5-bb30957a5145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:35.487515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535151a1-f1fb-4bbf-aad5-bb30957a5145 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:35.487799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535151a1-f1fb-4bbf-aad5-bb30957a5145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:35.488076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535151a1-f1fb-4bbf-aad5-bb30957a5145 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:35.488757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535151a1-f1fb-4bbf-aad5-bb30957a5145 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:35.490783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535151a1-f1fb-4bbf-aad5-bb30957a5145 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:35.543629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f7bfd-1ac7-4087-a5a9-99d7a9bb2d79 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:35.545028 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f7bfd-1ac7-4087-a5a9-99d7a9bb2d79 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:35.545448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f7bfd-1ac7-4087-a5a9-99d7a9bb2d79 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:35.547743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 839/1676] 10.222.0.39 () {54 vars in 912 bytes} [Tue Jul 26 07:25:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:25:35.553312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e83a3e-7fa3-46f3-bd85-e9c066171ece None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:35.554252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e83a3e-7fa3-46f3-bd85-e9c066171ece None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:35.555116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e83a3e-7fa3-46f3-bd85-e9c066171ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:35.568954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 838/1677] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:35 2022] POST /identity/v3/projects => generated 369 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:35.570589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535151a1-f1fb-4bbf-aad5-bb30957a5145 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:35.668584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b0e83a3e-7fa3-46f3-bd85-e9c066171ece None None] MFA Rules not processed for user `1911678a39f749939dc8476617a884fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:35.669894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0e83a3e-7fa3-46f3-bd85-e9c066171ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:35.732973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:35.740054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535151a1-f1fb-4bbf-aad5-bb30957a5145 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:35.755348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c216f8-4f9d-47e9-ae1e-619c1443ae6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:35.756080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c216f8-4f9d-47e9-ae1e-619c1443ae6a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:35.756432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c216f8-4f9d-47e9-ae1e-619c1443ae6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:35.756772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c216f8-4f9d-47e9-ae1e-619c1443ae6a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:35.760584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c216f8-4f9d-47e9-ae1e-619c1443ae6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:35.763779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c216f8-4f9d-47e9-ae1e-619c1443ae6a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:35.786026 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 840/1678] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:25:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:25:35.812875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e83a3e-7fa3-46f3-bd85-e9c066171ece None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:35.814908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0e83a3e-7fa3-46f3-bd85-e9c066171ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:35.830605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e83a3e-7fa3-46f3-bd85-e9c066171ece None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:35.836340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a087d-7819-4a7d-ac7f-3f063a470986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:35.837162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a087d-7819-4a7d-ac7f-3f063a470986 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:35.837712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a087d-7819-4a7d-ac7f-3f063a470986 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:35.838220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a087d-7819-4a7d-ac7f-3f063a470986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:35.848056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 839/1679] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:35 2022] POST /identity/v3/users => generated 291 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:35.848460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a087d-7819-4a7d-ac7f-3f063a470986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:35.849707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e303a-6b46-45cc-88ba-9562e00fe588 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:35.850730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a087d-7819-4a7d-ac7f-3f063a470986 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:35.856218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e303a-6b46-45cc-88ba-9562e00fe588 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:35.856512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e303a-6b46-45cc-88ba-9562e00fe588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:35.864583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 841/1680] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:35.865943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a087d-7819-4a7d-ac7f-3f063a470986 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:35.909899 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a087d-7819-4a7d-ac7f-3f063a470986 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:35.929871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b33f4-ecd8-4ec7-8e76-031350f160a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:35.931574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b33f4-ecd8-4ec7-8e76-031350f160a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:35.932846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b33f4-ecd8-4ec7-8e76-031350f160a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:35.933332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4b33f4-ecd8-4ec7-8e76-031350f160a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:35.935019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b33f4-ecd8-4ec7-8e76-031350f160a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:35.937176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4b33f4-ecd8-4ec7-8e76-031350f160a9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:35.954412 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6d0e303a-6b46-45cc-88ba-9562e00fe588 None None] MFA Rules not processed for user `08c4ec6a688c4608aa75aaf046933eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:35.962447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6d0e303a-6b46-45cc-88ba-9562e00fe588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:35.982414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 842/1681] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:35.986715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b33f4-ecd8-4ec7-8e76-031350f160a9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:36.060213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4b33f4-ecd8-4ec7-8e76-031350f160a9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:36.114736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4d4b5-3835-4945-9bb0-352874bf9f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:36.115738 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4d4b5-3835-4945-9bb0-352874bf9f43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:36.116165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4d4b5-3835-4945-9bb0-352874bf9f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:36.116577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4d4b5-3835-4945-9bb0-352874bf9f43 admin admin] PATH_INFO: `/v3/projects/7cbdd96e73a54b309d6626359cfdfd6a/users/4612e6e099bd4c25af8108a06f240f14/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:36.181468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4d4b5-3835-4945-9bb0-352874bf9f43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbdd96e73a54b309d6626359cfdfd6a, user_id=4612e6e099bd4c25af8108a06f240f14, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:36.190634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4d4b5-3835-4945-9bb0-352874bf9f43 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:36.261641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 843/1682] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:35 2022] PUT /identity/v3/projects/7cbdd96e73a54b309d6626359cfdfd6a/users/4612e6e099bd4c25af8108a06f240f14/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:36.263901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 840/1683] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:36.266275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e303a-6b46-45cc-88ba-9562e00fe588 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:36.292021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e303a-6b46-45cc-88ba-9562e00fe588 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:36.440891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4d4b5-3835-4945-9bb0-352874bf9f43 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:36.443685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bef4d4b5-3835-4945-9bb0-352874bf9f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:36.540503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becca34d-f7b3-4f35-ab35-a9b304289914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:36.541416 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becca34d-f7b3-4f35-ab35-a9b304289914 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:36.541781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becca34d-f7b3-4f35-ab35-a9b304289914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:36.541978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becca34d-f7b3-4f35-ab35-a9b304289914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:36.542745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becca34d-f7b3-4f35-ab35-a9b304289914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:36.548783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becca34d-f7b3-4f35-ab35-a9b304289914 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:36.601059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 841/1684] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:36.602552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becca34d-f7b3-4f35-ab35-a9b304289914 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:36.630781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becca34d-f7b3-4f35-ab35-a9b304289914 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:36.648669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e17dc-6c3d-4690-aa64-27508c2bc3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:36.653707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e17dc-6c3d-4690-aa64-27508c2bc3da admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:36.653707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e17dc-6c3d-4690-aa64-27508c2bc3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:36.656589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e17dc-6c3d-4690-aa64-27508c2bc3da admin admin] PATH_INFO: `/v3/projects/7cbdd96e73a54b309d6626359cfdfd6a/users/4612e6e099bd4c25af8108a06f240f14/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:36.662813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e17dc-6c3d-4690-aa64-27508c2bc3da admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cbdd96e73a54b309d6626359cfdfd6a, user_id=4612e6e099bd4c25af8108a06f240f14, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:36.670331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e17dc-6c3d-4690-aa64-27508c2bc3da admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:36.729358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 842/1685] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:36 2022] PUT /identity/v3/projects/7cbdd96e73a54b309d6626359cfdfd6a/users/4612e6e099bd4c25af8108a06f240f14/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:36.731178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e6b4-45da-4901-ac4c-af066b6a506f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:36.731464 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e6b4-45da-4901-ac4c-af066b6a506f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:36.731739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e6b4-45da-4901-ac4c-af066b6a506f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:36.738974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4d4b5-3835-4945-9bb0-352874bf9f43 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:36.857839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-14f7e6b4-45da-4901-ac4c-af066b6a506f None None] MFA Rules not processed for user `4612e6e099bd4c25af8108a06f240f14`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:36.859134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-14f7e6b4-45da-4901-ac4c-af066b6a506f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:36.877182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:37.012682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440a64e-03bd-4790-a01a-a29c4103d387 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:37.013456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440a64e-03bd-4790-a01a-a29c4103d387 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:37.013770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440a64e-03bd-4790-a01a-a29c4103d387 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:37.014344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440a64e-03bd-4790-a01a-a29c4103d387 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:37.034606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:37.104467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 843/1686] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:36 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:37.266349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7e6b4-45da-4901-ac4c-af066b6a506f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:37.270789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-14f7e6b4-45da-4901-ac4c-af066b6a506f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:37.277671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440a64e-03bd-4790-a01a-a29c4103d387 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:37.279375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440a64e-03bd-4790-a01a-a29c4103d387 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:37.330828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 844/1687] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 889 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:37.333619 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c459e3-fa0e-48f8-9aa7-cc7f81940c4d None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:37.333951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c459e3-fa0e-48f8-9aa7-cc7f81940c4d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:37.334233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c459e3-fa0e-48f8-9aa7-cc7f81940c4d None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:37.339119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 845/1688] 10.222.0.39 () {54 vars in 961 bytes} [Tue Jul 26 07:25:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:25:37.343498 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c4bb3-e32e-4d1d-b121-cb7eb289d1e0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:37.343789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c4bb3-e32e-4d1d-b121-cb7eb289d1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:37.344066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c4bb3-e32e-4d1d-b121-cb7eb289d1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:37.409777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-067c4bb3-e32e-4d1d-b121-cb7eb289d1e0 None None] MFA Rules not processed for user `51a5093a60ef45919ab6e159adf58336`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:37.414630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-067c4bb3-e32e-4d1d-b121-cb7eb289d1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:37.519959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7e6b4-45da-4901-ac4c-af066b6a506f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:37.536164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4eae9d-87ad-4ddf-83ac-0e04a63aa434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:37.536996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4eae9d-87ad-4ddf-83ac-0e04a63aa434 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:37.537386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4eae9d-87ad-4ddf-83ac-0e04a63aa434 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:37.537853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4eae9d-87ad-4ddf-83ac-0e04a63aa434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:37.574549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4eae9d-87ad-4ddf-83ac-0e04a63aa434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:37.576230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4eae9d-87ad-4ddf-83ac-0e04a63aa434 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:37.602271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 844/1689] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:37 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:37.603511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 846/1690] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Jul 26 07:25:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:37.609785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c4bb3-e32e-4d1d-b121-cb7eb289d1e0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:37.680205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c4bb3-e32e-4d1d-b121-cb7eb289d1e0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:37.702820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76832525-ed97-43b2-be98-f92b015d9501 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:37.703752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76832525-ed97-43b2-be98-f92b015d9501 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:37.704141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76832525-ed97-43b2-be98-f92b015d9501 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:37.704523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76832525-ed97-43b2-be98-f92b015d9501 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:37.710707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-76832525-ed97-43b2-be98-f92b015d9501 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:37.730128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76832525-ed97-43b2-be98-f92b015d9501 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:37.735129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76832525-ed97-43b2-be98-f92b015d9501 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:37.766571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 847/1691] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:37.837976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4eae9d-87ad-4ddf-83ac-0e04a63aa434 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:37.849673 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4eae9d-87ad-4ddf-83ac-0e04a63aa434 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:37.867794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6019a3-267f-4d10-a1fb-017742b64aad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:37.871780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6019a3-267f-4d10-a1fb-017742b64aad service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:37.873866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6019a3-267f-4d10-a1fb-017742b64aad service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:37.874326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6019a3-267f-4d10-a1fb-017742b64aad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:38.085983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6019a3-267f-4d10-a1fb-017742b64aad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:38.087510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6019a3-267f-4d10-a1fb-017742b64aad service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:38.128133 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 845/1692] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:38.131359 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76832525-ed97-43b2-be98-f92b015d9501 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:38.136802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-76832525-ed97-43b2-be98-f92b015d9501 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:38.424119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:38.491576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76832525-ed97-43b2-be98-f92b015d9501 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:38.513863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:38.515732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:38.516418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:38.516886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] PATH_INFO: `/v3/users/1a58299139db4f02beca97cd8feed5f9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:38.521058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a58299139db4f02beca97cd8feed5f9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:38.525634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: WARNING py.warnings [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 26 07:25:38.525634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: warnings.warn(msg) Jul 26 07:25:38.525634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: Jul 26 07:25:38.527154 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:38.756380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a58299139db4f02beca97cd8feed5f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:38.757101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a58299139db4f02beca97cd8feed5f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:38.854552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a58299139db4f02beca97cd8feed5f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:38.884114 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a58299139db4f02beca97cd8feed5f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:38.906319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a58299139db4f02beca97cd8feed5f9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:38.916475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 848/1693] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:38 2022] DELETE /identity/v3/users/1a58299139db4f02beca97cd8feed5f9 => generated 0 bytes in 786 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:38.922566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3394bbb-f867-41d9-9456-54328cd4b6fd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:38.923308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3394bbb-f867-41d9-9456-54328cd4b6fd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:38.923308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3394bbb-f867-41d9-9456-54328cd4b6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:38.983752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a3394bbb-f867-41d9-9456-54328cd4b6fd None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:38.985016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3394bbb-f867-41d9-9456-54328cd4b6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:39.169136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 846/1694] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:39.330247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:39.332431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:39.388165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3394bbb-f867-41d9-9456-54328cd4b6fd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:39.389932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3394bbb-f867-41d9-9456-54328cd4b6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:39.445032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3394bbb-f867-41d9-9456-54328cd4b6fd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:39.468472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:39.473182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:39.473610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:39.475106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] PATH_INFO: `/v3/users/cfb4f20954d34751a96beb40be86300a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:39.480234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb4f20954d34751a96beb40be86300a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:39.492772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:39.580202 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a9864f-541d-417a-aac3-9dcde68cd7d6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:39.585017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:39.597997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c969b-f37e-4262-b908-20c8b165543b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:39.601461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c969b-f37e-4262-b908-20c8b165543b service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:39.601802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c969b-f37e-4262-b908-20c8b165543b service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:39.604969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c969b-f37e-4262-b908-20c8b165543b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:39.646520 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c969b-f37e-4262-b908-20c8b165543b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:39.651583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c969b-f37e-4262-b908-20c8b165543b service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:39.669907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb4f20954d34751a96beb40be86300a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:39.678911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb4f20954d34751a96beb40be86300a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:39.695372 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 849/1695] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:39.720848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb4f20954d34751a96beb40be86300a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:39.737121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb4f20954d34751a96beb40be86300a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:39.752658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb4f20954d34751a96beb40be86300a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:39.764940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 847/1696] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:39 2022] DELETE /identity/v3/users/cfb4f20954d34751a96beb40be86300a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:25:39.770150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264fa70-c72f-4597-9989-31bbbcb1d420 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:39.770504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264fa70-c72f-4597-9989-31bbbcb1d420 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:39.770740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4264fa70-c72f-4597-9989-31bbbcb1d420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:39.812134 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4264fa70-c72f-4597-9989-31bbbcb1d420 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:39.817037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4264fa70-c72f-4597-9989-31bbbcb1d420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:39.927131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 850/1697] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:40.040177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:40.042134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:40.180687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b1935-8605-4476-9b14-9b90d6a98a1c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:40.181018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b1935-8605-4476-9b14-9b90d6a98a1c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:40.181251 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b1935-8605-4476-9b14-9b90d6a98a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:40.264918 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:40.276696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c01b1935-8605-4476-9b14-9b90d6a98a1c None None] MFA Rules not processed for user `886151888fc84ccf890d609ba62da9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:40.283827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c01b1935-8605-4476-9b14-9b90d6a98a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:40.302824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e10cc8-0a08-45c9-bffb-2b8fe6dfbca2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:40.320691 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb538be-382c-4e0f-aa36-271b1402ada8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:40.322545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb538be-382c-4e0f-aa36-271b1402ada8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:40.322839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb538be-382c-4e0f-aa36-271b1402ada8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:40.326348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb538be-382c-4e0f-aa36-271b1402ada8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:40.419514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb538be-382c-4e0f-aa36-271b1402ada8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:40.428195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb538be-382c-4e0f-aa36-271b1402ada8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:40.486695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 848/1698] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:40.516235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb538be-382c-4e0f-aa36-271b1402ada8 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:40.518123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ceb538be-382c-4e0f-aa36-271b1402ada8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:40.556275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb538be-382c-4e0f-aa36-271b1402ada8 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:40.582083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:40.582779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:40.583068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:40.585118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] PATH_INFO: `/v3/users/9e87e1a57f8844fb8ad7b3fabd95a5ac` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:40.590702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e87e1a57f8844fb8ad7b3fabd95a5ac)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:40.598183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:40.661362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 851/1699] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:40.663163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876adcbc-edfb-463f-93ae-08107b580511 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:40.663448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876adcbc-edfb-463f-93ae-08107b580511 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:40.663947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876adcbc-edfb-463f-93ae-08107b580511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:40.755458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-876adcbc-edfb-463f-93ae-08107b580511 None None] MFA Rules not processed for user `886151888fc84ccf890d609ba62da9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:40.761261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-876adcbc-edfb-463f-93ae-08107b580511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:40.806286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 852/1700] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:40.815784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e87e1a57f8844fb8ad7b3fabd95a5ac'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:40.817582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876adcbc-edfb-463f-93ae-08107b580511 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:40.818172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e87e1a57f8844fb8ad7b3fabd95a5ac'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:40.822292 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-876adcbc-edfb-463f-93ae-08107b580511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:40.893926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e87e1a57f8844fb8ad7b3fabd95a5ac'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:40.920992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e87e1a57f8844fb8ad7b3fabd95a5ac'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:40.981800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b5d8601a-ed6c-47d4-bda1-a26135e8c177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e87e1a57f8844fb8ad7b3fabd95a5ac'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:41.015914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 849/1701] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:40 2022] DELETE /identity/v3/users/9e87e1a57f8844fb8ad7b3fabd95a5ac => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:41.020817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c610b-c1e0-49dc-a1b5-35858b503d8e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:41.021521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c610b-c1e0-49dc-a1b5-35858b503d8e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:41.023345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c610b-c1e0-49dc-a1b5-35858b503d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:41.074676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-155c610b-c1e0-49dc-a1b5-35858b503d8e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:41.080293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-155c610b-c1e0-49dc-a1b5-35858b503d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:41.113461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:41.249147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876adcbc-edfb-463f-93ae-08107b580511 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:41.337400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 850/1702] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:41.347949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3f96e-1991-4a53-815a-75521f0325dc None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:41.348343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3f96e-1991-4a53-815a-75521f0325dc None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:41.349934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f3f96e-1991-4a53-815a-75521f0325dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:41.430893 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a9f3f96e-1991-4a53-815a-75521f0325dc None None] MFA Rules not processed for user `8fdc64e58cb041f98d9ea4b8c8cd9e60`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:41.439593 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a9f3f96e-1991-4a53-815a-75521f0325dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:41.521690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b76a5-d59f-4e78-83b6-fde564688c5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:41.522542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b76a5-d59f-4e78-83b6-fde564688c5b service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:41.522830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b76a5-d59f-4e78-83b6-fde564688c5b service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:41.523108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b76a5-d59f-4e78-83b6-fde564688c5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:41.532989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-045b76a5-d59f-4e78-83b6-fde564688c5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:41.632444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 851/1703] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:41.633882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3f96e-1991-4a53-815a-75521f0325dc None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:41.635855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a9f3f96e-1991-4a53-815a-75521f0325dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:41.678040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:41.752781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b76a5-d59f-4e78-83b6-fde564688c5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:41.754885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b76a5-d59f-4e78-83b6-fde564688c5b service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:41.767212 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 853/1704] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 950 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:25:41.768418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafe82b-e3bc-49de-b8d1-fab5da72eaf9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:41.768787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafe82b-e3bc-49de-b8d1-fab5da72eaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:41.769075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aafe82b-e3bc-49de-b8d1-fab5da72eaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:41.817970 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1aafe82b-e3bc-49de-b8d1-fab5da72eaf9 None None] MFA Rules not processed for user `8fdc64e58cb041f98d9ea4b8c8cd9e60`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:41.822286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1aafe82b-e3bc-49de-b8d1-fab5da72eaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:41.836630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f3f96e-1991-4a53-815a-75521f0325dc None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:41.876943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 854/1705] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:41.877799 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafe82b-e3bc-49de-b8d1-fab5da72eaf9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:41.879550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa04737-4d12-4a4e-beba-4f292799b567 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:41.880192 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa04737-4d12-4a4e-beba-4f292799b567 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:41.880498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa04737-4d12-4a4e-beba-4f292799b567 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:41.880744 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa04737-4d12-4a4e-beba-4f292799b567 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:41.886008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1aafe82b-e3bc-49de-b8d1-fab5da72eaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:41.951376 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa04737-4d12-4a4e-beba-4f292799b567 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:41.952933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa04737-4d12-4a4e-beba-4f292799b567 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:41.986218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aafe82b-e3bc-49de-b8d1-fab5da72eaf9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:42.008801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:42.009490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:42.009815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:42.010071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] PATH_INFO: `/v3/projects/9c39dec587e24c84b3c9702e177a2dbd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:42.012560 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 852/1706] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:42.013595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa04737-4d12-4a4e-beba-4f292799b567 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:42.016073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c39dec587e24c84b3c9702e177a2dbd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:42.019366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:42.022025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2aa04737-4d12-4a4e-beba-4f292799b567 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:42.077340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa04737-4d12-4a4e-beba-4f292799b567 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:42.091144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d51260-e4e7-4ae4-82a9-0cade6a64f8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:42.092127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d51260-e4e7-4ae4-82a9-0cade6a64f8e service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:42.092622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d51260-e4e7-4ae4-82a9-0cade6a64f8e service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:42.093051 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d51260-e4e7-4ae4-82a9-0cade6a64f8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:42.112207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d51260-e4e7-4ae4-82a9-0cade6a64f8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:42.127800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d51260-e4e7-4ae4-82a9-0cade6a64f8e service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:42.162085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 853/1707] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:42.163210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d51260-e4e7-4ae4-82a9-0cade6a64f8e service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:42.167614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f5d51260-e4e7-4ae4-82a9-0cade6a64f8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:42.169889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c39dec587e24c84b3c9702e177a2dbd', 'request_id': 'req-d49fb3d1-a9bd-464c-9239-31034d957972'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:42.186636 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c39dec587e24c84b3c9702e177a2dbd', 'request_id': 'req-d49fb3d1-a9bd-464c-9239-31034d957972'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:42.197809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d51260-e4e7-4ae4-82a9-0cade6a64f8e service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:42.206201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c39dec587e24c84b3c9702e177a2dbd', 'request_id': 'req-d49fb3d1-a9bd-464c-9239-31034d957972'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:42.229863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] The token cache is being invalidate because project 9c39dec587e24c84b3c9702e177a2dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:25:42.231016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:42.232646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d49fb3d1-a9bd-464c-9239-31034d957972 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:42.236834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 855/1708] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:25:41 2022] DELETE /identity/v3/projects/9c39dec587e24c84b3c9702e177a2dbd => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:42.241959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9f6c7-159a-4ee8-ad9f-3db0f501ac08 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:42.242391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9f6c7-159a-4ee8-ad9f-3db0f501ac08 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:42.243642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9f6c7-159a-4ee8-ad9f-3db0f501ac08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:42.279803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:42.285066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c9c9f6c7-159a-4ee8-ad9f-3db0f501ac08 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:42.292107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c9c9f6c7-159a-4ee8-ad9f-3db0f501ac08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:42.369479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:42.370358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:42.370643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:42.370919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] PATH_INFO: `/v3/projects/df0856fb51f6430bb262c1c6aa6625f5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:42.374036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0856fb51f6430bb262c1c6aa6625f5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:42.378993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:42.473685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 856/1709] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:42.473685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a6108-3a3a-4242-905d-e7ae8fe5c1f0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:42.473685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a6108-3a3a-4242-905d-e7ae8fe5c1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:42.473685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a6108-3a3a-4242-905d-e7ae8fe5c1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:42.514550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0856fb51f6430bb262c1c6aa6625f5', 'request_id': 'req-43195126-892e-4d72-a670-41f58d266310'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:42.534992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0856fb51f6430bb262c1c6aa6625f5', 'request_id': 'req-43195126-892e-4d72-a670-41f58d266310'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:42.552610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0856fb51f6430bb262c1c6aa6625f5', 'request_id': 'req-43195126-892e-4d72-a670-41f58d266310'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:42.593676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cd5a6108-3a3a-4242-905d-e7ae8fe5c1f0 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:42.595379 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd5a6108-3a3a-4242-905d-e7ae8fe5c1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:42.599383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] The token cache is being invalidate because project df0856fb51f6430bb262c1c6aa6625f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:25:42.601022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:42.601955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-43195126-892e-4d72-a670-41f58d266310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:42.610446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 854/1710] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:25:42 2022] DELETE /identity/v3/projects/df0856fb51f6430bb262c1c6aa6625f5 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:42.613848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4929c0-e763-4b00-afbe-b4a0382bbe87 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:42.614156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4929c0-e763-4b00-afbe-b4a0382bbe87 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:42.614471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4929c0-e763-4b00-afbe-b4a0382bbe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:42.642548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:42.668202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cb4929c0-e763-4b00-afbe-b4a0382bbe87 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:42.671792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb4929c0-e763-4b00-afbe-b4a0382bbe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:42.755625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 857/1711] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:25:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:42.756588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a6108-3a3a-4242-905d-e7ae8fe5c1f0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:42.760815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cd5a6108-3a3a-4242-905d-e7ae8fe5c1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:42.830688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 855/1712] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:42.832089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4929c0-e763-4b00-afbe-b4a0382bbe87 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:42.852515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4929c0-e763-4b00-afbe-b4a0382bbe87 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:42.872148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f5384-07ad-4606-ba8c-06adbd31de70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:42.872687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5384-07ad-4606-ba8c-06adbd31de70 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:42.873067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5384-07ad-4606-ba8c-06adbd31de70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:42.873479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7f5384-07ad-4606-ba8c-06adbd31de70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:42.874690 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f5384-07ad-4606-ba8c-06adbd31de70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:42.875329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f5384-07ad-4606-ba8c-06adbd31de70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:42.877115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7f5384-07ad-4606-ba8c-06adbd31de70 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:42.917658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 856/1713] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:42.923301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f5384-07ad-4606-ba8c-06adbd31de70 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:42.937925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a6108-3a3a-4242-905d-e7ae8fe5c1f0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:42.942324 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7f5384-07ad-4606-ba8c-06adbd31de70 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:42.948562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:42.949525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:42.950594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:42.951244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] PATH_INFO: `/v3/users/e8254bf5fabb43fabef2931437b41971` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:42.953639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8254bf5fabb43fabef2931437b41971)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:42.957415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5acf9f-42be-4e63-9954-edf4c9f50b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:42.959407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:42.960219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5acf9f-42be-4e63-9954-edf4c9f50b8d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:42.960796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5acf9f-42be-4e63-9954-edf4c9f50b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:42.960920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5acf9f-42be-4e63-9954-edf4c9f50b8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:42.964258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5acf9f-42be-4e63-9954-edf4c9f50b8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:42.966014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5acf9f-42be-4e63-9954-edf4c9f50b8d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:43.041697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 857/1714] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:42 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:43.049891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5acf9f-42be-4e63-9954-edf4c9f50b8d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:43.116881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8254bf5fabb43fabef2931437b41971'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:43.118512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8254bf5fabb43fabef2931437b41971'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:43.147722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8254bf5fabb43fabef2931437b41971'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:43.163600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8254bf5fabb43fabef2931437b41971'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:43.182226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8254bf5fabb43fabef2931437b41971'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:43.213965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 858/1715] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:42 2022] DELETE /identity/v3/users/e8254bf5fabb43fabef2931437b41971 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:43.225668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:43.227660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:43.231399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:43.333255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5acf9f-42be-4e63-9954-edf4c9f50b8d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:43.388535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:43.589217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cc495-1720-4dfd-9121-9bb7b2f58660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:43.591538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cc495-1720-4dfd-9121-9bb7b2f58660 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:43.591859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cc495-1720-4dfd-9121-9bb7b2f58660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:43.592237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cc495-1720-4dfd-9121-9bb7b2f58660 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:43.592859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cc495-1720-4dfd-9121-9bb7b2f58660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:43.596367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cc495-1720-4dfd-9121-9bb7b2f58660 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:43.598718 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462a35ae-a30e-4ac1-b25e-4af6b7e0cb53 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:43.613061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e4d65-2238-427b-b09f-521a22f6eead service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:43.614004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e4d65-2238-427b-b09f-521a22f6eead service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:43.614418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e4d65-2238-427b-b09f-521a22f6eead service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:43.614796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e4d65-2238-427b-b09f-521a22f6eead service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:43.619435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-492e4d65-2238-427b-b09f-521a22f6eead service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:43.646762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 858/1716] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:43 2022] POST /identity/v3/users => generated 284 bytes in 596 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:25:43.648369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cc495-1720-4dfd-9121-9bb7b2f58660 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:43.653634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-838cc495-1720-4dfd-9121-9bb7b2f58660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:43.687195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cc495-1720-4dfd-9121-9bb7b2f58660 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:43.706172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36120986-9b5e-4c4d-bc7e-39bf41fd6c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:43.707008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36120986-9b5e-4c4d-bc7e-39bf41fd6c29 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:43.707430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36120986-9b5e-4c4d-bc7e-39bf41fd6c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:43.708034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36120986-9b5e-4c4d-bc7e-39bf41fd6c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:43.709567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36120986-9b5e-4c4d-bc7e-39bf41fd6c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:43.716719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36120986-9b5e-4c4d-bc7e-39bf41fd6c29 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:43.747402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 859/1717] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:43.749789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36120986-9b5e-4c4d-bc7e-39bf41fd6c29 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:43.791400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36120986-9b5e-4c4d-bc7e-39bf41fd6c29 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:43.808495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585b912-53b0-43d9-b231-8557dae412b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:43.809516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585b912-53b0-43d9-b231-8557dae412b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:43.810219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585b912-53b0-43d9-b231-8557dae412b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:43.810602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585b912-53b0-43d9-b231-8557dae412b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:43.811794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e4d65-2238-427b-b09f-521a22f6eead service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:43.813137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585b912-53b0-43d9-b231-8557dae412b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:43.814956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585b912-53b0-43d9-b231-8557dae412b9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:43.816346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e4d65-2238-427b-b09f-521a22f6eead service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:43.837155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 859/1718] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:25:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:25:43.841554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0065ae-248f-413f-a674-cb8a3b5c501a None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:43.841554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0065ae-248f-413f-a674-cb8a3b5c501a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:43.842171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0065ae-248f-413f-a674-cb8a3b5c501a None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:43.843415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 860/1719] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:43.853624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585b912-53b0-43d9-b231-8557dae412b9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:43.854449 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 860/1720] 10.222.0.39 () {54 vars in 912 bytes} [Tue Jul 26 07:25:43 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:25:43.855995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830552-ffb7-446b-abcd-ee37b4388850 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:43.860539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830552-ffb7-446b-abcd-ee37b4388850 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:43.861240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64830552-ffb7-446b-abcd-ee37b4388850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:43.887855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585b912-53b0-43d9-b231-8557dae412b9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:43.914554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2a9ce-4170-4127-97f4-2e8c6c37de37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:43.917556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2a9ce-4170-4127-97f4-2e8c6c37de37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:43.927435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2a9ce-4170-4127-97f4-2e8c6c37de37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:43.930454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2a9ce-4170-4127-97f4-2e8c6c37de37 admin admin] PATH_INFO: `/v3/projects/56da8d5935264cdf9b072dbdbe6976b0/users/2811b3ac0e3042cda3029e9173d06d72/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:43.942091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-64830552-ffb7-446b-abcd-ee37b4388850 None None] MFA Rules not processed for user `1911678a39f749939dc8476617a884fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:43.947108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-64830552-ffb7-446b-abcd-ee37b4388850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:43.975297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2a9ce-4170-4127-97f4-2e8c6c37de37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56da8d5935264cdf9b072dbdbe6976b0, user_id=2811b3ac0e3042cda3029e9173d06d72, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:43.980769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2a9ce-4170-4127-97f4-2e8c6c37de37 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:44.028221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 861/1721] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:43 2022] PUT /identity/v3/projects/56da8d5935264cdf9b072dbdbe6976b0/users/2811b3ac0e3042cda3029e9173d06d72/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:44.039873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc35b7-57df-4362-bc36-8afe540de085 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:44.040314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc35b7-57df-4362-bc36-8afe540de085 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:44.040718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc35b7-57df-4362-bc36-8afe540de085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:44.147226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6cdc35b7-57df-4362-bc36-8afe540de085 None None] MFA Rules not processed for user `2811b3ac0e3042cda3029e9173d06d72`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:44.147932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 861/1722] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:25:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:44.149787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6cdc35b7-57df-4362-bc36-8afe540de085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:44.179042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64830552-ffb7-446b-abcd-ee37b4388850 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:44.180686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-64830552-ffb7-446b-abcd-ee37b4388850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:44.203042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64830552-ffb7-446b-abcd-ee37b4388850 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:44.395861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 862/1723] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:25:44.397513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc35b7-57df-4362-bc36-8afe540de085 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:44.415510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9de72-6323-4565-8ae7-71d63a957e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:44.421781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9de72-6323-4565-8ae7-71d63a957e83 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:44.422461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9de72-6323-4565-8ae7-71d63a957e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:44.424642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f9de72-6323-4565-8ae7-71d63a957e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:44.428810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:44.488140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9de72-6323-4565-8ae7-71d63a957e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:44.490399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f9de72-6323-4565-8ae7-71d63a957e83 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:44.546276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 862/1724] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:25:44.610633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc35b7-57df-4362-bc36-8afe540de085 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:44.621448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e28692-1726-49a6-b63b-2061580b76ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:44.624048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28692-1726-49a6-b63b-2061580b76ff admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:44.625255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28692-1726-49a6-b63b-2061580b76ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:44.625820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e28692-1726-49a6-b63b-2061580b76ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:44.629062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e28692-1726-49a6-b63b-2061580b76ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:44.633591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e28692-1726-49a6-b63b-2061580b76ff admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:44.670886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9de72-6323-4565-8ae7-71d63a957e83 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:44.673214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-31f9de72-6323-4565-8ae7-71d63a957e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:44.701621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f9de72-6323-4565-8ae7-71d63a957e83 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:44.703850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 863/1725] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:44 2022] POST /identity/v3/projects => generated 365 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:44.708056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e28692-1726-49a6-b63b-2061580b76ff admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:44.727715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:44.925365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e28692-1726-49a6-b63b-2061580b76ff admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:44.947334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8533a9-3fed-46e8-87fd-f23af65ce848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:44.948129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8533a9-3fed-46e8-87fd-f23af65ce848 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:44.948516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8533a9-3fed-46e8-87fd-f23af65ce848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:44.948897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8533a9-3fed-46e8-87fd-f23af65ce848 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:44.949911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8533a9-3fed-46e8-87fd-f23af65ce848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:44.954145 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8533a9-3fed-46e8-87fd-f23af65ce848 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:44.956041 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:44.956735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:44.957023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:44.957304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] PATH_INFO: `/v3/users/72f981865d484de8877fdbbe300799bb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:44.964075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f981865d484de8877fdbbe300799bb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:44.970384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:45.008919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 864/1726] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:44 2022] POST /identity/v3/users => generated 289 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:45.010910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8533a9-3fed-46e8-87fd-f23af65ce848 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:45.036990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8533a9-3fed-46e8-87fd-f23af65ce848 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:45.049621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120954-5ee2-46e3-8a44-885b4d547961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:45.050543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120954-5ee2-46e3-8a44-885b4d547961 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:45.050762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120954-5ee2-46e3-8a44-885b4d547961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:45.051102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120954-5ee2-46e3-8a44-885b4d547961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:45.051961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd120954-5ee2-46e3-8a44-885b4d547961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:45.056705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd120954-5ee2-46e3-8a44-885b4d547961 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:45.083443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 865/1727] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:45.088257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120954-5ee2-46e3-8a44-885b4d547961 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:45.108930 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120954-5ee2-46e3-8a44-885b4d547961 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:45.143442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b6a53-b7dd-426f-9e4b-95bb8cbe19e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:45.144178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b6a53-b7dd-426f-9e4b-95bb8cbe19e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:45.144425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b6a53-b7dd-426f-9e4b-95bb8cbe19e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:45.144700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b6a53-b7dd-426f-9e4b-95bb8cbe19e1 admin admin] PATH_INFO: `/v3/projects/a9d208079cc247958ff01e9c4b0fb56f/users/6397dd37477a4e2f919f54f022c14bc5/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:45.160723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f981865d484de8877fdbbe300799bb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:45.166052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f981865d484de8877fdbbe300799bb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:45.193344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b6a53-b7dd-426f-9e4b-95bb8cbe19e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d208079cc247958ff01e9c4b0fb56f, user_id=6397dd37477a4e2f919f54f022c14bc5, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:45.195126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f981865d484de8877fdbbe300799bb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:45.196127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b6a53-b7dd-426f-9e4b-95bb8cbe19e1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:45.208272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f981865d484de8877fdbbe300799bb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:45.224512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 866/1728] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:45 2022] PUT /identity/v3/projects/a9d208079cc247958ff01e9c4b0fb56f/users/6397dd37477a4e2f919f54f022c14bc5/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:45.227693 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b6a53-b7dd-426f-9e4b-95bb8cbe19e1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:45.235520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c75b6a53-b7dd-426f-9e4b-95bb8cbe19e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:45.251484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f981865d484de8877fdbbe300799bb'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:45.270127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 863/1729] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:44 2022] DELETE /identity/v3/users/72f981865d484de8877fdbbe300799bb => generated 0 bytes in 599 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:45.272166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:45.275387 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:45.430900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:45.462766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b6a53-b7dd-426f-9e4b-95bb8cbe19e1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:45.475686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:45.478549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:45.480230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:45.481081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] PATH_INFO: `/v3/projects/8a8e01f61251484c970e66ba5c4ec0b3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:45.488060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a8e01f61251484c970e66ba5c4ec0b3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:45.488942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c65099-738d-4524-acaa-888c69f3ce3e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:45.494851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:45.502629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426e698-d1ae-429d-aef4-0381b57fbdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:45.505301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426e698-d1ae-429d-aef4-0381b57fbdbe admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:45.508126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426e698-d1ae-429d-aef4-0381b57fbdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:45.510357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426e698-d1ae-429d-aef4-0381b57fbdbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:45.511867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426e698-d1ae-429d-aef4-0381b57fbdbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:45.519767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426e698-d1ae-429d-aef4-0381b57fbdbe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:45.561069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 864/1730] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:45.565162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8ba68-ca10-42d7-9105-3c7943722705 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:45.566746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8ba68-ca10-42d7-9105-3c7943722705 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:45.567854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b8ba68-ca10-42d7-9105-3c7943722705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:45.627265 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-66b8ba68-ca10-42d7-9105-3c7943722705 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:45.631119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66b8ba68-ca10-42d7-9105-3c7943722705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:45.660359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a8e01f61251484c970e66ba5c4ec0b3', 'request_id': 'req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:45.680491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a8e01f61251484c970e66ba5c4ec0b3', 'request_id': 'req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:45.710974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a8e01f61251484c970e66ba5c4ec0b3', 'request_id': 'req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:45.771208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:45.788990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] The token cache is being invalidate because project 8a8e01f61251484c970e66ba5c4ec0b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:25:45.789166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:45.791814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:45.802866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 867/1731] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:25:45 2022] DELETE /identity/v3/projects/8a8e01f61251484c970e66ba5c4ec0b3 => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:45.803935 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:45.817076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:45.946233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 865/1732] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:45.950408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8ba68-ca10-42d7-9105-3c7943722705 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:45.959475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66b8ba68-ca10-42d7-9105-3c7943722705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:46.031400 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b8ba68-ca10-42d7-9105-3c7943722705 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:46.054116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c029cf-95d1-48f7-b1e3-ebb78ddcad29 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:46.062723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69265c84-b7e8-4e67-9b97-f8efeebe237a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:46.063979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69265c84-b7e8-4e67-9b97-f8efeebe237a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:46.064484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69265c84-b7e8-4e67-9b97-f8efeebe237a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:46.065453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69265c84-b7e8-4e67-9b97-f8efeebe237a admin admin] PATH_INFO: `/v3/projects/a9d208079cc247958ff01e9c4b0fb56f/users/6397dd37477a4e2f919f54f022c14bc5/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:46.068828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69265c84-b7e8-4e67-9b97-f8efeebe237a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d208079cc247958ff01e9c4b0fb56f, user_id=6397dd37477a4e2f919f54f022c14bc5, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:46.072936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69265c84-b7e8-4e67-9b97-f8efeebe237a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:46.084033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:46.084770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:46.092852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:46.093376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] PATH_INFO: `/v3/projects/711f0e36d3a04f62bb410e01dd445053` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:46.104686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 868/1733] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:45 2022] PUT /identity/v3/projects/a9d208079cc247958ff01e9c4b0fb56f/users/6397dd37477a4e2f919f54f022c14bc5/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:46.107179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=711f0e36d3a04f62bb410e01dd445053)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:46.109180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:46.115469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c8207-c13d-4c73-afaa-dabfeb791857 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:46.115808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c8207-c13d-4c73-afaa-dabfeb791857 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:46.118555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c8207-c13d-4c73-afaa-dabfeb791857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:46.166300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9e9c8207-c13d-4c73-afaa-dabfeb791857 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:46.167446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9e9c8207-c13d-4c73-afaa-dabfeb791857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:46.338195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '711f0e36d3a04f62bb410e01dd445053', 'request_id': 'req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:46.361791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 869/1734] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:46.363062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be2a16-9ae3-48da-97e5-ce4ab3a07e15 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:46.363347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be2a16-9ae3-48da-97e5-ce4ab3a07e15 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:46.363620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be2a16-9ae3-48da-97e5-ce4ab3a07e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:46.365370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '711f0e36d3a04f62bb410e01dd445053', 'request_id': 'req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:46.388233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '711f0e36d3a04f62bb410e01dd445053', 'request_id': 'req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:46.422238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] The token cache is being invalidate because project 711f0e36d3a04f62bb410e01dd445053 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:25:46.422537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:46.443931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:46.444577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:46.453921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 866/1735] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:25:45 2022] DELETE /identity/v3/projects/711f0e36d3a04f62bb410e01dd445053 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:46.459736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c2be2a16-9ae3-48da-97e5-ce4ab3a07e15 None None] MFA Rules not processed for user `6397dd37477a4e2f919f54f022c14bc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:46.460597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:46.466476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:46.473097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2be2a16-9ae3-48da-97e5-ce4ab3a07e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:46.639840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811ac0f-f9f1-427f-8c23-6dd94a11dab8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:46.652491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1a7f1-582c-4d63-bd6a-9511aae60ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:46.653189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1a7f1-582c-4d63-bd6a-9511aae60ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:46.653526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1a7f1-582c-4d63-bd6a-9511aae60ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:46.653908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1a7f1-582c-4d63-bd6a-9511aae60ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:46.655946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c5f1a7f1-582c-4d63-bd6a-9511aae60ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:46.853545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 870/1736] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 491 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:46.857201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03dbde-9c86-4f8e-b4a9-7ae0d61fda0b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:46.857201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03dbde-9c86-4f8e-b4a9-7ae0d61fda0b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:46.857390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03dbde-9c86-4f8e-b4a9-7ae0d61fda0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:46.934285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4e03dbde-9c86-4f8e-b4a9-7ae0d61fda0b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:46.939731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4e03dbde-9c86-4f8e-b4a9-7ae0d61fda0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:46.950676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1a7f1-582c-4d63-bd6a-9511aae60ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:46.956268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1a7f1-582c-4d63-bd6a-9511aae60ba6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:46.967872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 871/1737] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:46.968752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03dbde-9c86-4f8e-b4a9-7ae0d61fda0b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:46.972427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4e03dbde-9c86-4f8e-b4a9-7ae0d61fda0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:46.991807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 867/1738] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:46.994375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1a7f1-582c-4d63-bd6a-9511aae60ba6 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:47.001801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03dbde-9c86-4f8e-b4a9-7ae0d61fda0b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:47.011276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:47.012136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:47.012419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:47.013909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] PATH_INFO: `/v3/users/57b3f112691e441884e31971fd67e0ab` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:47.016769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57b3f112691e441884e31971fd67e0ab)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:47.024287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:47.025129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c5f1a7f1-582c-4d63-bd6a-9511aae60ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:47.053457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:47.136579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57b3f112691e441884e31971fd67e0ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:47.139284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57b3f112691e441884e31971fd67e0ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:47.169492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57b3f112691e441884e31971fd67e0ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:47.203889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57b3f112691e441884e31971fd67e0ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:47.246094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57b3f112691e441884e31971fd67e0ab'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:47.266479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 872/1739] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:46 2022] DELETE /identity/v3/users/57b3f112691e441884e31971fd67e0ab => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:47.267658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:47.275053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:47.290299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1a7f1-582c-4d63-bd6a-9511aae60ba6 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:47.450749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a147303-3452-41b2-8524-4316dcb74722 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:47.461283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b45a781-5a53-46b7-9c6d-3c959b7a7999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:47.462367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45a781-5a53-46b7-9c6d-3c959b7a7999 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:47.463126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45a781-5a53-46b7-9c6d-3c959b7a7999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:47.463605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45a781-5a53-46b7-9c6d-3c959b7a7999 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:47.464649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45a781-5a53-46b7-9c6d-3c959b7a7999 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:47.465186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45a781-5a53-46b7-9c6d-3c959b7a7999 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:47.467247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b45a781-5a53-46b7-9c6d-3c959b7a7999 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:47.469096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058ea48-0778-43e7-8527-40a56c52de5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:47.471754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058ea48-0778-43e7-8527-40a56c52de5f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:47.473620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058ea48-0778-43e7-8527-40a56c52de5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:47.473967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058ea48-0778-43e7-8527-40a56c52de5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:47.480631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c058ea48-0778-43e7-8527-40a56c52de5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:47.482134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:47.487150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 873/1740] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:47.489324 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146f76a-330f-45bc-9170-05d05503b4ab None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:47.489971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146f76a-330f-45bc-9170-05d05503b4ab None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:47.490471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7146f76a-330f-45bc-9170-05d05503b4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:47.518278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7146f76a-330f-45bc-9170-05d05503b4ab None None] MFA Rules not processed for user `886151888fc84ccf890d609ba62da9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:47.521301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7146f76a-330f-45bc-9170-05d05503b4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:47.627791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 874/1741] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:47.633298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146f76a-330f-45bc-9170-05d05503b4ab None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:47.664224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7146f76a-330f-45bc-9170-05d05503b4ab None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:47.727621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6fa17-7221-4b3d-a911-9b196522b82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:47.729167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6fa17-7221-4b3d-a911-9b196522b82b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:47.730291 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058ea48-0778-43e7-8527-40a56c52de5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:47.730752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6fa17-7221-4b3d-a911-9b196522b82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:47.732051 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6fa17-7221-4b3d-a911-9b196522b82b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:47.734418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c058ea48-0778-43e7-8527-40a56c52de5f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:47.735275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6fa17-7221-4b3d-a911-9b196522b82b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:47.742344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6fa17-7221-4b3d-a911-9b196522b82b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:47.759848 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 868/1742] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 767 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:47.908900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058ea48-0778-43e7-8527-40a56c52de5f service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:47.910208 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 875/1743] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:47 2022] POST /identity/v3/projects => generated 357 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:47.930815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058ea48-0778-43e7-8527-40a56c52de5f service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:48.377658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b6373-725c-4aee-adcd-f023073c8aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:48.380121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b6373-725c-4aee-adcd-f023073c8aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:48.383631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b6373-725c-4aee-adcd-f023073c8aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:48.384208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b6373-725c-4aee-adcd-f023073c8aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:48.389233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b6373-725c-4aee-adcd-f023073c8aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:48.399399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1b6373-725c-4aee-adcd-f023073c8aa5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:48.458514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:48.484476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 869/1744] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:47 2022] POST /identity/v3/users => generated 279 bytes in 576 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:48.487537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6fa17-7221-4b3d-a911-9b196522b82b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:48.501393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6fa17-7221-4b3d-a911-9b196522b82b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:48.521617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369bd15-44c9-4506-9ad4-88138d42a0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:48.539121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369bd15-44c9-4506-9ad4-88138d42a0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:48.539627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369bd15-44c9-4506-9ad4-88138d42a0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:48.540378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a369bd15-44c9-4506-9ad4-88138d42a0e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:48.542708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369bd15-44c9-4506-9ad4-88138d42a0e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:48.544663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a369bd15-44c9-4506-9ad4-88138d42a0e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:48.572391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 876/1745] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:48.577698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b6373-725c-4aee-adcd-f023073c8aa5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:48.610664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b6373-725c-4aee-adcd-f023073c8aa5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:48.644122 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7b8c5-b2ba-41b5-870b-ca50f89978fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:48.646793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7b8c5-b2ba-41b5-870b-ca50f89978fc admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:48.647326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7b8c5-b2ba-41b5-870b-ca50f89978fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:48.648346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7b8c5-b2ba-41b5-870b-ca50f89978fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:48.652334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7b8c5-b2ba-41b5-870b-ca50f89978fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:48.658250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a7b8c5-b2ba-41b5-870b-ca50f89978fc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:48.690342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 870/1746] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:48.692745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369bd15-44c9-4506-9ad4-88138d42a0e8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:48.714292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a369bd15-44c9-4506-9ad4-88138d42a0e8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:48.729978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa54d5dc-fc63-4f36-b35c-c289bc8b3321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:48.733074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54d5dc-fc63-4f36-b35c-c289bc8b3321 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:48.737492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54d5dc-fc63-4f36-b35c-c289bc8b3321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:48.738483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa54d5dc-fc63-4f36-b35c-c289bc8b3321 admin admin] PATH_INFO: `/v3/projects/16c8c18654c249f08075779ceb4c4fc0/users/a2d77afdefec41bebe2eb3012d927aae/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:48.829477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54d5dc-fc63-4f36-b35c-c289bc8b3321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c8c18654c249f08075779ceb4c4fc0, user_id=a2d77afdefec41bebe2eb3012d927aae, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:48.849375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa54d5dc-fc63-4f36-b35c-c289bc8b3321 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:48.880804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 877/1747] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:48 2022] PUT /identity/v3/projects/16c8c18654c249f08075779ceb4c4fc0/users/a2d77afdefec41bebe2eb3012d927aae/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:48.901267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742a75-345d-43b1-bac0-4c0189dedfa6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:48.902469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742a75-345d-43b1-bac0-4c0189dedfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:48.902858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35742a75-345d-43b1-bac0-4c0189dedfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:48.984980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-35742a75-345d-43b1-bac0-4c0189dedfa6 None None] MFA Rules not processed for user `a2d77afdefec41bebe2eb3012d927aae`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:48.989283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35742a75-345d-43b1-bac0-4c0189dedfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:49.141341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee451c7-e386-468f-9d9e-e0dfd2a1ea9c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:49.141954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee451c7-e386-468f-9d9e-e0dfd2a1ea9c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:49.142410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee451c7-e386-468f-9d9e-e0dfd2a1ea9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:49.196881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8ee451c7-e386-468f-9d9e-e0dfd2a1ea9c None None] MFA Rules not processed for user `1911678a39f749939dc8476617a884fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:49.198190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8ee451c7-e386-468f-9d9e-e0dfd2a1ea9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:49.367108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 871/1748] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:48 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:49.376362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35742a75-345d-43b1-bac0-4c0189dedfa6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:49.419288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 878/1749] 10.222.0.43 () {58 vars in 1015 bytes} [Tue Jul 26 07:25:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:25:49.573251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4349ae-94ab-49c1-9509-93458e0a6ede None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:49.573965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4349ae-94ab-49c1-9509-93458e0a6ede None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:49.574476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4349ae-94ab-49c1-9509-93458e0a6ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:49.660972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35742a75-345d-43b1-bac0-4c0189dedfa6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:49.693708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014307b-3353-4368-8ffe-30669fb06f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:49.694485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014307b-3353-4368-8ffe-30669fb06f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:49.694782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014307b-3353-4368-8ffe-30669fb06f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:49.695216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4014307b-3353-4368-8ffe-30669fb06f34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:49.695787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014307b-3353-4368-8ffe-30669fb06f34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:49.712883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4014307b-3353-4368-8ffe-30669fb06f34 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:49.713901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ce4349ae-94ab-49c1-9509-93458e0a6ede None None] MFA Rules not processed for user `08c4ec6a688c4608aa75aaf046933eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:49.722466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ce4349ae-94ab-49c1-9509-93458e0a6ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:49.847982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 872/1750] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:49 2022] POST /identity/v3/projects => generated 357 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:25:49.849014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014307b-3353-4368-8ffe-30669fb06f34 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:49.862335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4014307b-3353-4368-8ffe-30669fb06f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:49.967385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:49.996825 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 879/1751] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 425 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:49.997553 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4349ae-94ab-49c1-9509-93458e0a6ede None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:50.036244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4349ae-94ab-49c1-9509-93458e0a6ede None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:50.065138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4014307b-3353-4368-8ffe-30669fb06f34 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:50.075959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e01f1-072b-4820-a27a-e67ccbc962c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:50.080056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e01f1-072b-4820-a27a-e67ccbc962c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:50.081587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e01f1-072b-4820-a27a-e67ccbc962c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:50.081943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e01f1-072b-4820-a27a-e67ccbc962c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:50.106472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e01f1-072b-4820-a27a-e67ccbc962c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:50.108012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3e01f1-072b-4820-a27a-e67ccbc962c6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:50.151537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 873/1752] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:50.229882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf8bd6-c071-4229-a339-ba87d71a4149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:50.230882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf8bd6-c071-4229-a339-ba87d71a4149 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:50.231366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf8bd6-c071-4229-a339-ba87d71a4149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:50.232444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf8bd6-c071-4229-a339-ba87d71a4149 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:50.234653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf8bd6-c071-4229-a339-ba87d71a4149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:50.242196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf8bd6-c071-4229-a339-ba87d71a4149 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:50.298472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e01f1-072b-4820-a27a-e67ccbc962c6 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:50.308695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:50.309999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 880/1753] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:49 2022] POST /identity/v3/users => generated 285 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:50.317040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e01f1-072b-4820-a27a-e67ccbc962c6 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:50.332516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4899a-4c13-4a85-b036-5944674f5518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:50.333277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4899a-4c13-4a85-b036-5944674f5518 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:50.333511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4899a-4c13-4a85-b036-5944674f5518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:50.333808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4899a-4c13-4a85-b036-5944674f5518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:50.334535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4899a-4c13-4a85-b036-5944674f5518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:50.336186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4899a-4c13-4a85-b036-5944674f5518 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:50.369223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 874/1754] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:50.373328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf8bd6-c071-4229-a339-ba87d71a4149 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:50.397099 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf8bd6-c071-4229-a339-ba87d71a4149 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:50.402541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7b110-846d-4ee2-9305-eaed8fc03a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:50.403259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7b110-846d-4ee2-9305-eaed8fc03a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:50.403576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7b110-846d-4ee2-9305-eaed8fc03a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:50.403865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7b110-846d-4ee2-9305-eaed8fc03a47 admin admin] PATH_INFO: `/v3/projects/aeeac2d781e94d5d9b9bc3a7820f5f41/users/d53ad1397e084613b500892ae9f34e7d/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:50.448735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7b110-846d-4ee2-9305-eaed8fc03a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeeac2d781e94d5d9b9bc3a7820f5f41, user_id=d53ad1397e084613b500892ae9f34e7d, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:50.453569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a7b110-846d-4ee2-9305-eaed8fc03a47 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:50.488188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 881/1755] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:50 2022] PUT /identity/v3/projects/aeeac2d781e94d5d9b9bc3a7820f5f41/users/d53ad1397e084613b500892ae9f34e7d/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:50.493660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4899a-4c13-4a85-b036-5944674f5518 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:50.510989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4899a-4c13-4a85-b036-5944674f5518 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:50.807954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d117e5f-e758-4ba8-b6bd-501675459f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:50.808967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117e5f-e758-4ba8-b6bd-501675459f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:50.810130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117e5f-e758-4ba8-b6bd-501675459f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:50.811035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d117e5f-e758-4ba8-b6bd-501675459f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:50.812201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d117e5f-e758-4ba8-b6bd-501675459f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:50.815525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d117e5f-e758-4ba8-b6bd-501675459f59 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:50.886407 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7b110-846d-4ee2-9305-eaed8fc03a47 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:50.902370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 875/1756] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:25:50.918104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d117e5f-e758-4ba8-b6bd-501675459f59 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:50.921597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1d117e5f-e758-4ba8-b6bd-501675459f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:50.927768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7b110-846d-4ee2-9305-eaed8fc03a47 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:50.945711 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e8ecb-7544-4a1a-9b23-f0a06c8d46d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:50.946891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e8ecb-7544-4a1a-9b23-f0a06c8d46d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:50.947692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e8ecb-7544-4a1a-9b23-f0a06c8d46d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:50.956750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e8ecb-7544-4a1a-9b23-f0a06c8d46d5 admin admin] PATH_INFO: `/v3/projects/aeeac2d781e94d5d9b9bc3a7820f5f41/users/d53ad1397e084613b500892ae9f34e7d/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:50.965287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e8ecb-7544-4a1a-9b23-f0a06c8d46d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeeac2d781e94d5d9b9bc3a7820f5f41, user_id=d53ad1397e084613b500892ae9f34e7d, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:50.971426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e8ecb-7544-4a1a-9b23-f0a06c8d46d5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:51.020150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d117e5f-e758-4ba8-b6bd-501675459f59 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:51.026464 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 882/1757] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:50 2022] PUT /identity/v3/projects/aeeac2d781e94d5d9b9bc3a7820f5f41/users/d53ad1397e084613b500892ae9f34e7d/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:51.040042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f25942-d742-4fc9-a0a8-b4a8fcda0c1d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:51.040611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f25942-d742-4fc9-a0a8-b4a8fcda0c1d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:51.040871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f25942-d742-4fc9-a0a8-b4a8fcda0c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:51.158481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-59f25942-d742-4fc9-a0a8-b4a8fcda0c1d None None] MFA Rules not processed for user `d53ad1397e084613b500892ae9f34e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:51.159975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-59f25942-d742-4fc9-a0a8-b4a8fcda0c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:51.318744 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:51.320882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:51.321167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:51.321444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] PATH_INFO: `/v3/projects/19b0687625c44767bbca6729ab4dcd76` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:51.325994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19b0687625c44767bbca6729ab4dcd76)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:51.332096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:51.460552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 883/1758] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:51 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:25:51.468437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f25942-d742-4fc9-a0a8-b4a8fcda0c1d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:51.470808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-59f25942-d742-4fc9-a0a8-b4a8fcda0c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:51.531385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19b0687625c44767bbca6729ab4dcd76', 'request_id': 'req-0c08be47-6815-41c6-9fed-40a42e44eda3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:51.555134 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19b0687625c44767bbca6729ab4dcd76', 'request_id': 'req-0c08be47-6815-41c6-9fed-40a42e44eda3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:51.571195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19b0687625c44767bbca6729ab4dcd76', 'request_id': 'req-0c08be47-6815-41c6-9fed-40a42e44eda3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:51.572409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:51.604092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] The token cache is being invalidate because project 19b0687625c44767bbca6729ab4dcd76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:25:51.604402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:51.606731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:51.608205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f25942-d742-4fc9-a0a8-b4a8fcda0c1d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:51.613086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 876/1759] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:25:50 2022] DELETE /identity/v3/projects/19b0687625c44767bbca6729ab4dcd76 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:51.614285 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:51.616129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:51.660080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:51.724238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4a622-213d-4253-a640-3e971dc0ed54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:51.727090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4a622-213d-4253-a640-3e971dc0ed54 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:51.727090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4a622-213d-4253-a640-3e971dc0ed54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:51.727090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4a622-213d-4253-a640-3e971dc0ed54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:51.733866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ade4a622-213d-4253-a640-3e971dc0ed54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:51.792616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08be47-6815-41c6-9fed-40a42e44eda3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:51.798892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:51.799995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:51.799995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:51.800213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] PATH_INFO: `/v3/users/4612e6e099bd4c25af8108a06f240f14` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:51.802586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4612e6e099bd4c25af8108a06f240f14)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:51.806950 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:51.912575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4a622-213d-4253-a640-3e971dc0ed54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:51.915423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4612e6e099bd4c25af8108a06f240f14'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:51.916198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4612e6e099bd4c25af8108a06f240f14'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:51.918882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4a622-213d-4253-a640-3e971dc0ed54 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:51.945763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 884/1760] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:51 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:25:51.946652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4a622-213d-4253-a640-3e971dc0ed54 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:51.948044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ade4a622-213d-4253-a640-3e971dc0ed54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:51.950311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4612e6e099bd4c25af8108a06f240f14'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:51.971266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4a622-213d-4253-a640-3e971dc0ed54 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:51.976201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4612e6e099bd4c25af8108a06f240f14'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:51.988401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4612e6e099bd4c25af8108a06f240f14'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:51.990613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:51.991708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:51.992143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:51.992684 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] PATH_INFO: `/v3/users/2811b3ac0e3042cda3029e9173d06d72` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:51.994357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2811b3ac0e3042cda3029e9173d06d72)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:51.996785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:52.003509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 877/1761] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:51 2022] DELETE /identity/v3/users/4612e6e099bd4c25af8108a06f240f14 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:52.088391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:52.090322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:52.172485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2811b3ac0e3042cda3029e9173d06d72'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:52.173788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2811b3ac0e3042cda3029e9173d06d72'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:52.212107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2811b3ac0e3042cda3029e9173d06d72'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:52.232072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2811b3ac0e3042cda3029e9173d06d72'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:52.235644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37da76c7-f9c4-453c-9910-80f5ed16100c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:52.248672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4bec6-1327-4ef7-985e-fe993920ffbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:52.249637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4bec6-1327-4ef7-985e-fe993920ffbd service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:52.251659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4bec6-1327-4ef7-985e-fe993920ffbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:52.252105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4bec6-1327-4ef7-985e-fe993920ffbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:52.257271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bcb4bec6-1327-4ef7-985e-fe993920ffbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:52.266972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2811b3ac0e3042cda3029e9173d06d72'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:52.283229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 885/1762] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:51 2022] DELETE /identity/v3/users/2811b3ac0e3042cda3029e9173d06d72 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:52.289791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:52.304377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:52.362145 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:52.454834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:52.469771 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4bec6-1327-4ef7-985e-fe993920ffbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:52.471237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb4bec6-1327-4ef7-985e-fe993920ffbd service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:52.521453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 878/1763] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:52 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:25:52.524305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ce2d7-4b45-4f09-a24d-2b42baf315f8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:52.524627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ce2d7-4b45-4f09-a24d-2b42baf315f8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:52.524908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793ce2d7-4b45-4f09-a24d-2b42baf315f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:52.606112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-793ce2d7-4b45-4f09-a24d-2b42baf315f8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:52.607337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-793ce2d7-4b45-4f09-a24d-2b42baf315f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:52.629269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4905c23-63de-47d5-b5c1-dd86d3598f3f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:52.635472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 879/1764] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:52.640597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:52.642059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:52.642577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:52.643034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] PATH_INFO: `/v3/projects/e0ac581979c44d3eaa7fcfd9fb9382e4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:52.644895 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0ac581979c44d3eaa7fcfd9fb9382e4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:52.648080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:52.728332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0ac581979c44d3eaa7fcfd9fb9382e4', 'request_id': 'req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:52.747851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0ac581979c44d3eaa7fcfd9fb9382e4', 'request_id': 'req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:52.765681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0ac581979c44d3eaa7fcfd9fb9382e4', 'request_id': 'req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:52.796497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] The token cache is being invalidate because project e0ac581979c44d3eaa7fcfd9fb9382e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:25:52.796831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:52.799119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8939c6c0-cbe8-4e59-b9cb-9ca38b18e3d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:52.806610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 886/1765] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:25:52 2022] DELETE /identity/v3/projects/e0ac581979c44d3eaa7fcfd9fb9382e4 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:52.810007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ce2d7-4b45-4f09-a24d-2b42baf315f8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:52.811886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-793ce2d7-4b45-4f09-a24d-2b42baf315f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:52.816357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edcaa4-a89b-443d-97a3-73fab0d757b7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:52.816665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edcaa4-a89b-443d-97a3-73fab0d757b7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:52.816906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8edcaa4-a89b-443d-97a3-73fab0d757b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:52.923151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f8edcaa4-a89b-443d-97a3-73fab0d757b7 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:52.924399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f8edcaa4-a89b-443d-97a3-73fab0d757b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:53.012025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793ce2d7-4b45-4f09-a24d-2b42baf315f8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:53.156147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da60096-8890-48ba-8940-372920b3c89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:53.158077 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da60096-8890-48ba-8940-372920b3c89a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:53.158397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da60096-8890-48ba-8940-372920b3c89a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:53.158881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da60096-8890-48ba-8940-372920b3c89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:53.166770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3da60096-8890-48ba-8940-372920b3c89a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:53.179210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 887/1766] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:53.190569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb9762-9a0c-473c-becf-01d3fa0ce35e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:53.190875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb9762-9a0c-473c-becf-01d3fa0ce35e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:53.191125 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bb9762-9a0c-473c-becf-01d3fa0ce35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:53.210707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da60096-8890-48ba-8940-372920b3c89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:53.215547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da60096-8890-48ba-8940-372920b3c89a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:53.248661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 880/1767] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:53.272124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d1bb9762-9a0c-473c-becf-01d3fa0ce35e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:53.283386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d1bb9762-9a0c-473c-becf-01d3fa0ce35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:53.304877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 888/1768] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:53.312055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da60096-8890-48ba-8940-372920b3c89a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:53.336355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da60096-8890-48ba-8940-372920b3c89a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:53.358057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4b309-8be3-4906-b684-aa2935450398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:53.360534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4b309-8be3-4906-b684-aa2935450398 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:53.361666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4b309-8be3-4906-b684-aa2935450398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:53.363273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4b309-8be3-4906-b684-aa2935450398 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:53.366014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4b309-8be3-4906-b684-aa2935450398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:53.366357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4b309-8be3-4906-b684-aa2935450398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:53.376257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf4b309-8be3-4906-b684-aa2935450398 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:53.431227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb9762-9a0c-473c-becf-01d3fa0ce35e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:53.431830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 881/1769] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:53.443297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bb9762-9a0c-473c-becf-01d3fa0ce35e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:53.494825 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bc9e5-76ee-45f4-8950-5a5172f7f484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:53.495309 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bc9e5-76ee-45f4-8950-5a5172f7f484 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:53.495602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bc9e5-76ee-45f4-8950-5a5172f7f484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:53.495880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7bc9e5-76ee-45f4-8950-5a5172f7f484 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:53.496547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bc9e5-76ee-45f4-8950-5a5172f7f484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:53.504575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7bc9e5-76ee-45f4-8950-5a5172f7f484 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:53.608077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 889/1770] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:53 2022] POST /identity/v3/projects => generated 399 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:53.612683 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4b309-8be3-4906-b684-aa2935450398 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:53.626215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4b309-8be3-4906-b684-aa2935450398 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:53.786851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88100471-f812-4653-ade6-c708c6e3177a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:53.788016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88100471-f812-4653-ade6-c708c6e3177a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:53.788474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88100471-f812-4653-ade6-c708c6e3177a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:53.789078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88100471-f812-4653-ade6-c708c6e3177a admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:53.790421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88100471-f812-4653-ade6-c708c6e3177a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:53.793381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88100471-f812-4653-ade6-c708c6e3177a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:53.854333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 882/1771] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:53 2022] POST /identity/v3/users => generated 300 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:53.857590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bc9e5-76ee-45f4-8950-5a5172f7f484 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:53.872816 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7bc9e5-76ee-45f4-8950-5a5172f7f484 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:53.888976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ebe49-51cc-4569-8749-b25b73ed3774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:53.889769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ebe49-51cc-4569-8749-b25b73ed3774 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:53.890088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ebe49-51cc-4569-8749-b25b73ed3774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:53.890367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ebe49-51cc-4569-8749-b25b73ed3774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:53.891201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ebe49-51cc-4569-8749-b25b73ed3774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:53.892834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ebe49-51cc-4569-8749-b25b73ed3774 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:53.922331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 890/1772] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:53.925428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88100471-f812-4653-ade6-c708c6e3177a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:53.943009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88100471-f812-4653-ade6-c708c6e3177a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:53.956186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d3a77-425f-4284-a122-1872d0776cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:53.957107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d3a77-425f-4284-a122-1872d0776cef admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:53.957542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d3a77-425f-4284-a122-1872d0776cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:53.957991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d3a77-425f-4284-a122-1872d0776cef admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:53.963098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d3a77-425f-4284-a122-1872d0776cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:53.965198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d3a77-425f-4284-a122-1872d0776cef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:54.010237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 883/1773] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:25:54.014705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ebe49-51cc-4569-8749-b25b73ed3774 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:54.032302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ebe49-51cc-4569-8749-b25b73ed3774 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:54.050318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed5123-1934-4e65-b02f-d8e4c3e48709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:54.050318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed5123-1934-4e65-b02f-d8e4c3e48709 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:54.050318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed5123-1934-4e65-b02f-d8e4c3e48709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:54.050808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ed5123-1934-4e65-b02f-d8e4c3e48709 admin admin] PATH_INFO: `/v3/projects/f8a9a7e2d28e4f56855abfcb939eccee/users/555b58a0a8a04a60a25c96c19d956ab6/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:54.054028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:54.097280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed5123-1934-4e65-b02f-d8e4c3e48709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8a9a7e2d28e4f56855abfcb939eccee, user_id=555b58a0a8a04a60a25c96c19d956ab6, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:54.100267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ed5123-1934-4e65-b02f-d8e4c3e48709 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:54.130348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 891/1774] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:54 2022] PUT /identity/v3/projects/f8a9a7e2d28e4f56855abfcb939eccee/users/555b58a0a8a04a60a25c96c19d956ab6/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:54.142418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8849d78-f50d-4586-842f-21db06e6814d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:54.143352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8849d78-f50d-4586-842f-21db06e6814d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:54.143781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8849d78-f50d-4586-842f-21db06e6814d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:54.193890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:54.236147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a8849d78-f50d-4586-842f-21db06e6814d None None] MFA Rules not processed for user `555b58a0a8a04a60a25c96c19d956ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:54.239036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8849d78-f50d-4586-842f-21db06e6814d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:54.467646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 884/1775] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:54 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:54.472556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed5123-1934-4e65-b02f-d8e4c3e48709 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:54.662122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ed5123-1934-4e65-b02f-d8e4c3e48709 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:54.699225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84691480-b3c7-42bb-917d-4bf4bdddc7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:54.702108 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84691480-b3c7-42bb-917d-4bf4bdddc7cf admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:54.703923 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84691480-b3c7-42bb-917d-4bf4bdddc7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:54.704679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84691480-b3c7-42bb-917d-4bf4bdddc7cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:54.707166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84691480-b3c7-42bb-917d-4bf4bdddc7cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:54.714080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84691480-b3c7-42bb-917d-4bf4bdddc7cf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:54.798334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 892/1776] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:54 2022] POST /identity/v3/projects => generated 397 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:25:54.805405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8849d78-f50d-4586-842f-21db06e6814d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:54.839030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8849d78-f50d-4586-842f-21db06e6814d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:55.080839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec2715-e2be-4e7a-9126-f62a11b8183f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:55.085172 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec2715-e2be-4e7a-9126-f62a11b8183f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:55.087522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec2715-e2be-4e7a-9126-f62a11b8183f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:55.090568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec2715-e2be-4e7a-9126-f62a11b8183f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:55.092321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec2715-e2be-4e7a-9126-f62a11b8183f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:55.104704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec2715-e2be-4e7a-9126-f62a11b8183f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:55.199927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 885/1777] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:54 2022] POST /identity/v3/users => generated 305 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:55.200688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84691480-b3c7-42bb-917d-4bf4bdddc7cf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:55.264356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84691480-b3c7-42bb-917d-4bf4bdddc7cf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:55.305067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad4907-b782-478d-9e06-9204578c153a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:55.305722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad4907-b782-478d-9e06-9204578c153a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:55.306029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad4907-b782-478d-9e06-9204578c153a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:55.306300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad4907-b782-478d-9e06-9204578c153a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:55.326440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad4907-b782-478d-9e06-9204578c153a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:55.329163 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad4907-b782-478d-9e06-9204578c153a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:55.354869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 893/1778] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:25:55.359510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec2715-e2be-4e7a-9126-f62a11b8183f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:55.386526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec2715-e2be-4e7a-9126-f62a11b8183f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:55.393072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2c95c-2d51-4727-af76-a87e6518258e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:55.394222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2c95c-2d51-4727-af76-a87e6518258e admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:55.394362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2c95c-2d51-4727-af76-a87e6518258e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:55.394690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2c95c-2d51-4727-af76-a87e6518258e admin admin] PATH_INFO: `/v3/projects/702974231ce34b2d9cdc5dc48d4da831/users/3f3e204d9fb04cb4888d7e56bfd2b0e7/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:55.435251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908cbde-a4a1-48bb-83fa-ac02a9d4854e None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:55.435836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908cbde-a4a1-48bb-83fa-ac02a9d4854e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:55.436444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908cbde-a4a1-48bb-83fa-ac02a9d4854e None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:55.438311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2c95c-2d51-4727-af76-a87e6518258e admin admin] RBAC: Authorizing `identity:create_grant(project_id=702974231ce34b2d9cdc5dc48d4da831, user_id=3f3e204d9fb04cb4888d7e56bfd2b0e7, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:55.444968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 894/1779] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:25:55 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 26 07:25:55.455348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2c95c-2d51-4727-af76-a87e6518258e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:55.486440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 886/1780] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:55 2022] PUT /identity/v3/projects/702974231ce34b2d9cdc5dc48d4da831/users/3f3e204d9fb04cb4888d7e56bfd2b0e7/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:25:55.490225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908cbde-a4a1-48bb-83fa-ac02a9d4854e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:55.496341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389591d-b482-40c9-8cf6-877b45f8ebb8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:55.496632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389591d-b482-40c9-8cf6-877b45f8ebb8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:55.496897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389591d-b482-40c9-8cf6-877b45f8ebb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:55.539800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5389591d-b482-40c9-8cf6-877b45f8ebb8 None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:55.545484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5389591d-b482-40c9-8cf6-877b45f8ebb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:55.755096 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908cbde-a4a1-48bb-83fa-ac02a9d4854e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:55.765304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:55.793596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d90560-9ffb-461d-ba5b-2a0d5394cab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:55.797044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90560-9ffb-461d-ba5b-2a0d5394cab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:55.799573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90560-9ffb-461d-ba5b-2a0d5394cab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:55.799918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d90560-9ffb-461d-ba5b-2a0d5394cab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:55.800690 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d90560-9ffb-461d-ba5b-2a0d5394cab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:55.811489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d90560-9ffb-461d-ba5b-2a0d5394cab7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:55.813480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 887/1781] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:25:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:55.865400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 895/1782] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:25:55.870069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389591d-b482-40c9-8cf6-877b45f8ebb8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:55.890333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389591d-b482-40c9-8cf6-877b45f8ebb8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:55.915545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591ad83-b111-4125-9225-997969648884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:55.916366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591ad83-b111-4125-9225-997969648884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:55.916661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591ad83-b111-4125-9225-997969648884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:55.916939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591ad83-b111-4125-9225-997969648884 admin admin] PATH_INFO: `/v3/projects/702974231ce34b2d9cdc5dc48d4da831/users/3f3e204d9fb04cb4888d7e56bfd2b0e7/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:55.923809 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591ad83-b111-4125-9225-997969648884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702974231ce34b2d9cdc5dc48d4da831, user_id=3f3e204d9fb04cb4888d7e56bfd2b0e7, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:55.928186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591ad83-b111-4125-9225-997969648884 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:55.960834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 888/1783] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:25:55 2022] PUT /identity/v3/projects/702974231ce34b2d9cdc5dc48d4da831/users/3f3e204d9fb04cb4888d7e56bfd2b0e7/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:55.971972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f265e-abab-44fc-9c3d-83742d2bc857 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:55.972262 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f265e-abab-44fc-9c3d-83742d2bc857 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:55.972609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f265e-abab-44fc-9c3d-83742d2bc857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:56.011730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591ad83-b111-4125-9225-997969648884 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:56.013871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9591ad83-b111-4125-9225-997969648884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:56.025948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ac3f265e-abab-44fc-9c3d-83742d2bc857 None None] MFA Rules not processed for user `3f3e204d9fb04cb4888d7e56bfd2b0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:56.027196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac3f265e-abab-44fc-9c3d-83742d2bc857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:56.199318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:56.303983 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591ad83-b111-4125-9225-997969648884 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:56.322585 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003c9e0-961e-45bd-a3fc-13c36b75a355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:56.324714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003c9e0-961e-45bd-a3fc-13c36b75a355 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:56.326449 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003c9e0-961e-45bd-a3fc-13c36b75a355 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:56.328403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3003c9e0-961e-45bd-a3fc-13c36b75a355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:56.357423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003c9e0-961e-45bd-a3fc-13c36b75a355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:56.361405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3003c9e0-961e-45bd-a3fc-13c36b75a355 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:56.394026 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 889/1784] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:25:56.396548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003c9e0-961e-45bd-a3fc-13c36b75a355 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:56.400313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3003c9e0-961e-45bd-a3fc-13c36b75a355 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:56.444477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 896/1785] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:56.452621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f265e-abab-44fc-9c3d-83742d2bc857 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:56.459962 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ac3f265e-abab-44fc-9c3d-83742d2bc857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:56.592824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f265e-abab-44fc-9c3d-83742d2bc857 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:56.605557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafd0e9-fd1c-4454-ac5f-cfc08f2f9a79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:56.606434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafd0e9-fd1c-4454-ac5f-cfc08f2f9a79 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:56.606719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafd0e9-fd1c-4454-ac5f-cfc08f2f9a79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:56.606995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafd0e9-fd1c-4454-ac5f-cfc08f2f9a79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:56.635203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafd0e9-fd1c-4454-ac5f-cfc08f2f9a79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:56.639504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dafd0e9-fd1c-4454-ac5f-cfc08f2f9a79 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:56.643787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3003c9e0-961e-45bd-a3fc-13c36b75a355 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:56.658444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:56.659211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:56.659496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:56.659842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] PATH_INFO: `/v3/projects/56da8d5935264cdf9b072dbdbe6976b0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:56.661507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56da8d5935264cdf9b072dbdbe6976b0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:56.666512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:56.683609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 897/1786] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:25:56.805930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56da8d5935264cdf9b072dbdbe6976b0', 'request_id': 'req-76784b1c-a040-462f-a65d-78a3febbff79'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:56.822360 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56da8d5935264cdf9b072dbdbe6976b0', 'request_id': 'req-76784b1c-a040-462f-a65d-78a3febbff79'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:56.849092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56da8d5935264cdf9b072dbdbe6976b0', 'request_id': 'req-76784b1c-a040-462f-a65d-78a3febbff79'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:56.901177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] The token cache is being invalidate because project 56da8d5935264cdf9b072dbdbe6976b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:25:56.901578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:56.910142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:56.910142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 890/1787] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:25:56 2022] DELETE /identity/v3/projects/56da8d5935264cdf9b072dbdbe6976b0 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:56.910142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:56.912618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafd0e9-fd1c-4454-ac5f-cfc08f2f9a79 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:56.941144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafd0e9-fd1c-4454-ac5f-cfc08f2f9a79 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:56.949901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:56.951828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:57.110367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:57.111078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:57.111369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:57.111637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] PATH_INFO: `/v3/users/6397dd37477a4e2f919f54f022c14bc5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:57.116918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6397dd37477a4e2f919f54f022c14bc5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:57.119586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:57.132765 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76784b1c-a040-462f-a65d-78a3febbff79 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:57.141349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf836d-3005-4630-b621-5b4acc945e7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:57.141974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf836d-3005-4630-b621-5b4acc945e7c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:57.142421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf836d-3005-4630-b621-5b4acc945e7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:57.142708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdf836d-3005-4630-b621-5b4acc945e7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:57.148005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ecdf836d-3005-4630-b621-5b4acc945e7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:57.354227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6397dd37477a4e2f919f54f022c14bc5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:57.357268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6397dd37477a4e2f919f54f022c14bc5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:57.368856 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf836d-3005-4630-b621-5b4acc945e7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:57.375042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdf836d-3005-4630-b621-5b4acc945e7c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:57.401878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6397dd37477a4e2f919f54f022c14bc5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:57.408965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 891/1788] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:25:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:25:57.449373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6397dd37477a4e2f919f54f022c14bc5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:57.477455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-bba4f175-8c4c-45a9-a492-27880151a55a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6397dd37477a4e2f919f54f022c14bc5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:57.483802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:57.500834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 898/1789] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:25:56 2022] DELETE /identity/v3/users/6397dd37477a4e2f919f54f022c14bc5 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:58.871189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf836d-3005-4630-b621-5b4acc945e7c service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:58.872747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ecdf836d-3005-4630-b621-5b4acc945e7c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:58.895870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdf836d-3005-4630-b621-5b4acc945e7c service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:59.035183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:59.036343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:59.036860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:59.037383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] PATH_INFO: `/v3/projects/7cbdd96e73a54b309d6626359cfdfd6a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:59.039231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cbdd96e73a54b309d6626359cfdfd6a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:59.043158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:59.152386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cbdd96e73a54b309d6626359cfdfd6a', 'request_id': 'req-005ba556-3d20-4a8e-90cd-87748770199d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:59.165111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:25:59.167882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cbdd96e73a54b309d6626359cfdfd6a', 'request_id': 'req-005ba556-3d20-4a8e-90cd-87748770199d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:59.185149 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cbdd96e73a54b309d6626359cfdfd6a', 'request_id': 'req-005ba556-3d20-4a8e-90cd-87748770199d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:59.210411 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] The token cache is being invalidate because project 7cbdd96e73a54b309d6626359cfdfd6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:25:59.210877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:59.213425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-005ba556-3d20-4a8e-90cd-87748770199d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:25:59.215907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 892/1790] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:25:58 2022] DELETE /identity/v3/projects/7cbdd96e73a54b309d6626359cfdfd6a => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:25:59.222450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad4ed0-57d5-4c39-9fad-0944b2544b7d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:59.223042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad4ed0-57d5-4c39-9fad-0944b2544b7d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:59.223042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad4ed0-57d5-4c39-9fad-0944b2544b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:59.259713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-53ad4ed0-57d5-4c39-9fad-0944b2544b7d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:59.279917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-53ad4ed0-57d5-4c39-9fad-0944b2544b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:59.523934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 899/1791] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:59.532117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6a18-5cbb-488f-8d7c-ffa6f82c056c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:59.532725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6a18-5cbb-488f-8d7c-ffa6f82c056c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:59.533174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d6a18-5cbb-488f-8d7c-ffa6f82c056c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:59.579033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-713d6a18-5cbb-488f-8d7c-ffa6f82c056c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:25:59.580360 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-713d6a18-5cbb-488f-8d7c-ffa6f82c056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:25:59.610505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 893/1792] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:25:59.614503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad4ed0-57d5-4c39-9fad-0944b2544b7d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:59.638138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad4ed0-57d5-4c39-9fad-0944b2544b7d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:59.652413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a41f06-5638-4284-895a-59312a6235e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:59.653228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a41f06-5638-4284-895a-59312a6235e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:59.653606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a41f06-5638-4284-895a-59312a6235e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:59.654775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a41f06-5638-4284-895a-59312a6235e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:59.657377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a41f06-5638-4284-895a-59312a6235e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:25:59.687404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a41f06-5638-4284-895a-59312a6235e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:59.689360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a41f06-5638-4284-895a-59312a6235e3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:59.711717 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 900/1793] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:25:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:25:59.713822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6a18-5cbb-488f-8d7c-ffa6f82c056c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:59.725559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d6a18-5cbb-488f-8d7c-ffa6f82c056c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:25:59.752659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d94fc-bf22-4f27-b50e-67e04d9877c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:25:59.760562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d94fc-bf22-4f27-b50e-67e04d9877c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:25:59.761848 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d94fc-bf22-4f27-b50e-67e04d9877c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:25:59.763648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d94fc-bf22-4f27-b50e-67e04d9877c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:25:59.783424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d94fc-bf22-4f27-b50e-67e04d9877c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:25:59.789590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d94fc-bf22-4f27-b50e-67e04d9877c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:25:59.905236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 894/1794] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:25:59 2022] POST /identity/v3/projects => generated 365 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:25:59.909384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a41f06-5638-4284-895a-59312a6235e3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:25:59.934850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a41f06-5638-4284-895a-59312a6235e3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:00.086224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69576af-9da5-4c95-b742-601c52cf39f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:00.091068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69576af-9da5-4c95-b742-601c52cf39f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:00.091652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69576af-9da5-4c95-b742-601c52cf39f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:00.092116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69576af-9da5-4c95-b742-601c52cf39f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:00.093021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69576af-9da5-4c95-b742-601c52cf39f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:00.115294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69576af-9da5-4c95-b742-601c52cf39f0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:00.198685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 901/1795] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:25:59 2022] POST /identity/v3/users => generated 283 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:00.202961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d94fc-bf22-4f27-b50e-67e04d9877c2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:00.226243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d94fc-bf22-4f27-b50e-67e04d9877c2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:00.262682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da69a87-fa8c-48d4-9a83-60675c57899f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:00.266947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da69a87-fa8c-48d4-9a83-60675c57899f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:00.270813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da69a87-fa8c-48d4-9a83-60675c57899f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:00.272187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da69a87-fa8c-48d4-9a83-60675c57899f admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:00.273594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da69a87-fa8c-48d4-9a83-60675c57899f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:00.276795 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da69a87-fa8c-48d4-9a83-60675c57899f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:00.333423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 895/1796] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:00.334780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69576af-9da5-4c95-b742-601c52cf39f0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:00.354205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69576af-9da5-4c95-b742-601c52cf39f0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:00.360313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcfd71c-ffa4-47c1-ab50-958b93c19850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:00.360965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfd71c-ffa4-47c1-ab50-958b93c19850 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:00.361250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfd71c-ffa4-47c1-ab50-958b93c19850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:00.363437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcfd71c-ffa4-47c1-ab50-958b93c19850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:00.364310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfd71c-ffa4-47c1-ab50-958b93c19850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:00.371090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcfd71c-ffa4-47c1-ab50-958b93c19850 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:00.409624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 902/1797] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:00 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:00.414065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da69a87-fa8c-48d4-9a83-60675c57899f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:00.433588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da69a87-fa8c-48d4-9a83-60675c57899f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:00.442267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b220c20-7e7d-46e0-b27e-da98a5c8e412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:00.443386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b220c20-7e7d-46e0-b27e-da98a5c8e412 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:00.444389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b220c20-7e7d-46e0-b27e-da98a5c8e412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:00.444724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b220c20-7e7d-46e0-b27e-da98a5c8e412 admin admin] PATH_INFO: `/v3/projects/c23f83ba9d50403eac2fb8bc63dcb803/users/34d6259eb15e4dc4b9b6b4217aff41c6/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:00.478153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b220c20-7e7d-46e0-b27e-da98a5c8e412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c23f83ba9d50403eac2fb8bc63dcb803, user_id=34d6259eb15e4dc4b9b6b4217aff41c6, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:00.481691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b220c20-7e7d-46e0-b27e-da98a5c8e412 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:00.513370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 896/1798] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:00 2022] PUT /identity/v3/projects/c23f83ba9d50403eac2fb8bc63dcb803/users/34d6259eb15e4dc4b9b6b4217aff41c6/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:00.515542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18b2e0-c697-4dff-8ddf-6118c799cb68 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:00.515878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18b2e0-c697-4dff-8ddf-6118c799cb68 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:00.516158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18b2e0-c697-4dff-8ddf-6118c799cb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:00.556944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ea18b2e0-c697-4dff-8ddf-6118c799cb68 None None] MFA Rules not processed for user `34d6259eb15e4dc4b9b6b4217aff41c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:00.559055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ea18b2e0-c697-4dff-8ddf-6118c799cb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:00.631349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b220c20-7e7d-46e0-b27e-da98a5c8e412 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:00.649375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0b220c20-7e7d-46e0-b27e-da98a5c8e412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:00.685509 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:00.718778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 903/1799] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:00.721560 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18b2e0-c697-4dff-8ddf-6118c799cb68 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:00.828663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b220c20-7e7d-46e0-b27e-da98a5c8e412 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:00.840140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:00.840827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:00.841362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:00.841649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] PATH_INFO: `/v3/projects/a9d208079cc247958ff01e9c4b0fb56f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:00.844666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d208079cc247958ff01e9c4b0fb56f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:00.847135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:00.857560 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18b2e0-c697-4dff-8ddf-6118c799cb68 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:00.863643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817919d-d209-4943-9330-1665fbc714cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:00.864293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817919d-d209-4943-9330-1665fbc714cc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:00.864579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817919d-d209-4943-9330-1665fbc714cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:00.864857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817919d-d209-4943-9330-1665fbc714cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:00.865536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817919d-d209-4943-9330-1665fbc714cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:00.868268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817919d-d209-4943-9330-1665fbc714cc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:00.936903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d208079cc247958ff01e9c4b0fb56f', 'request_id': 'req-dce16304-e661-4153-9f1d-f7cd541e58d2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:00.938347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 904/1800] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:00 2022] POST /identity/v3/projects => generated 367 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:00.940093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1ce92-5377-47d3-965e-8db1e5205e1f None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:00.940217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1ce92-5377-47d3-965e-8db1e5205e1f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:00.940479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1ce92-5377-47d3-965e-8db1e5205e1f None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:00.942550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 905/1801] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:26:00.943702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1ce92-5377-47d3-965e-8db1e5205e1f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:00.949761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d208079cc247958ff01e9c4b0fb56f', 'request_id': 'req-dce16304-e661-4153-9f1d-f7cd541e58d2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:00.967117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d208079cc247958ff01e9c4b0fb56f', 'request_id': 'req-dce16304-e661-4153-9f1d-f7cd541e58d2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:00.983394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] The token cache is being invalidate because project a9d208079cc247958ff01e9c4b0fb56f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:00.984211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:00.985449 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-dce16304-e661-4153-9f1d-f7cd541e58d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:00.988091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 897/1802] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:00 2022] DELETE /identity/v3/projects/a9d208079cc247958ff01e9c4b0fb56f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:00.989974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea357e-4e66-4efe-9521-ad8f071fb6bf None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:00.990440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea357e-4e66-4efe-9521-ad8f071fb6bf None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:00.990883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebea357e-4e66-4efe-9521-ad8f071fb6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.042974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-ebea357e-4e66-4efe-9521-ad8f071fb6bf None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:01.044741 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ebea357e-4e66-4efe-9521-ad8f071fb6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:01.052509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:01.063079 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1ce92-5377-47d3-965e-8db1e5205e1f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.068486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e5d893-06ba-4777-b0b6-30cdea32f0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.069285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5d893-06ba-4777-b0b6-30cdea32f0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.069631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5d893-06ba-4777-b0b6-30cdea32f0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.070304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e5d893-06ba-4777-b0b6-30cdea32f0d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.071640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5d893-06ba-4777-b0b6-30cdea32f0d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.074835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e5d893-06ba-4777-b0b6-30cdea32f0d9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.107713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 906/1803] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:00 2022] POST /identity/v3/users => generated 290 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:01.107713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecac70-8e44-40c1-bae0-c46302aa54c9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.107713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecac70-8e44-40c1-bae0-c46302aa54c9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.107713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecac70-8e44-40c1-bae0-c46302aa54c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.134674 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dfecac70-8e44-40c1-bae0-c46302aa54c9 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:01.136517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dfecac70-8e44-40c1-bae0-c46302aa54c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:01.146445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 898/1804] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:26:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:01.148247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 907/1805] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:01.155181 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea357e-4e66-4efe-9521-ad8f071fb6bf None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.156775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ebea357e-4e66-4efe-9521-ad8f071fb6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:01.159465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a30e-d13e-4541-8f9f-d7edcb13c332 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.159802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a30e-d13e-4541-8f9f-d7edcb13c332 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.160369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35a30e-d13e-4541-8f9f-d7edcb13c332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.175587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebea357e-4e66-4efe-9521-ad8f071fb6bf None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.182030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82457581-f70c-43c1-b466-6b7f80986870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.183092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82457581-f70c-43c1-b466-6b7f80986870 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.184485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82457581-f70c-43c1-b466-6b7f80986870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.184926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82457581-f70c-43c1-b466-6b7f80986870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.186382 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82457581-f70c-43c1-b466-6b7f80986870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.188227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82457581-f70c-43c1-b466-6b7f80986870 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.197992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fd35a30e-d13e-4541-8f9f-d7edcb13c332 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:01.199209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fd35a30e-d13e-4541-8f9f-d7edcb13c332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:01.207308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 899/1806] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:01.208310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82457581-f70c-43c1-b466-6b7f80986870 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.209515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 908/1807] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:01.210567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35a30e-d13e-4541-8f9f-d7edcb13c332 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.215555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-82457581-f70c-43c1-b466-6b7f80986870 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:01.224468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35a30e-d13e-4541-8f9f-d7edcb13c332 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.229313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23253630-38dc-495c-b4db-0f86393cde2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.230044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23253630-38dc-495c-b4db-0f86393cde2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.230402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23253630-38dc-495c-b4db-0f86393cde2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.230587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23253630-38dc-495c-b4db-0f86393cde2f admin admin] PATH_INFO: `/v3/projects/24d07ef656e64fa4bd11dadf83447555/users/2d3fb63b8c5c47609045769877a312b4/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.249007 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23253630-38dc-495c-b4db-0f86393cde2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d07ef656e64fa4bd11dadf83447555, user_id=2d3fb63b8c5c47609045769877a312b4, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.253669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23253630-38dc-495c-b4db-0f86393cde2f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.262787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82457581-f70c-43c1-b466-6b7f80986870 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.275240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 909/1808] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:01 2022] PUT /identity/v3/projects/24d07ef656e64fa4bd11dadf83447555/users/2d3fb63b8c5c47609045769877a312b4/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:01.279980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23253630-38dc-495c-b4db-0f86393cde2f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.323201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:01.346308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb18305-931f-421d-a701-0fd92267b6b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.347067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb18305-931f-421d-a701-0fd92267b6b1 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.347357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb18305-931f-421d-a701-0fd92267b6b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.347857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb18305-931f-421d-a701-0fd92267b6b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.360515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb18305-931f-421d-a701-0fd92267b6b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.362172 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb18305-931f-421d-a701-0fd92267b6b1 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.363164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23253630-38dc-495c-b4db-0f86393cde2f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.368961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560dbd3-1855-4c8c-9c59-686b3fd0cacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.369786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560dbd3-1855-4c8c-9c59-686b3fd0cacb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.370149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560dbd3-1855-4c8c-9c59-686b3fd0cacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.370528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560dbd3-1855-4c8c-9c59-686b3fd0cacb admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.371904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560dbd3-1855-4c8c-9c59-686b3fd0cacb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:01.372310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560dbd3-1855-4c8c-9c59-686b3fd0cacb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.374182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560dbd3-1855-4c8c-9c59-686b3fd0cacb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.390035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 900/1809] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:01.390722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb18305-931f-421d-a701-0fd92267b6b1 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.402094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb18305-931f-421d-a701-0fd92267b6b1 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.408883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc11e99-9060-49a6-a341-9d9782e60d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.409525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc11e99-9060-49a6-a341-9d9782e60d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.411962 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc11e99-9060-49a6-a341-9d9782e60d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.412173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc11e99-9060-49a6-a341-9d9782e60d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.412889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc11e99-9060-49a6-a341-9d9782e60d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.414545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc11e99-9060-49a6-a341-9d9782e60d8e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.420925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 910/1810] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:01.425460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560dbd3-1855-4c8c-9c59-686b3fd0cacb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.443655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560dbd3-1855-4c8c-9c59-686b3fd0cacb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.445080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 901/1811] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:01.458299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc11e99-9060-49a6-a341-9d9782e60d8e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.473372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7321c-4932-4d83-846c-12ba1c15d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.474663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7321c-4932-4d83-846c-12ba1c15d3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.479196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7321c-4932-4d83-846c-12ba1c15d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.479616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7321c-4932-4d83-846c-12ba1c15d3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.482146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7321c-4932-4d83-846c-12ba1c15d3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.486084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7321c-4932-4d83-846c-12ba1c15d3e0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.494774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc11e99-9060-49a6-a341-9d9782e60d8e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.504626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be01b2-05a0-4113-9ba8-c41d2ddf4fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.505383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be01b2-05a0-4113-9ba8-c41d2ddf4fed admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.506222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be01b2-05a0-4113-9ba8-c41d2ddf4fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.506511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be01b2-05a0-4113-9ba8-c41d2ddf4fed admin admin] PATH_INFO: `/v3/projects/24d07ef656e64fa4bd11dadf83447555/users/2d3fb63b8c5c47609045769877a312b4/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.508828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be01b2-05a0-4113-9ba8-c41d2ddf4fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d07ef656e64fa4bd11dadf83447555, user_id=2d3fb63b8c5c47609045769877a312b4, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.511688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be01b2-05a0-4113-9ba8-c41d2ddf4fed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.537546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 902/1812] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:01 2022] PUT /identity/v3/projects/24d07ef656e64fa4bd11dadf83447555/users/2d3fb63b8c5c47609045769877a312b4/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:01.543045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea0a16-227f-479f-bb11-89bed6f51dd6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.543373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea0a16-227f-479f-bb11-89bed6f51dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.543647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea0a16-227f-479f-bb11-89bed6f51dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.561585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 911/1813] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:01 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:01.562888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7321c-4932-4d83-846c-12ba1c15d3e0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.631192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2eea0a16-227f-479f-bb11-89bed6f51dd6 None None] MFA Rules not processed for user `2d3fb63b8c5c47609045769877a312b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:01.632780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2eea0a16-227f-479f-bb11-89bed6f51dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:01.745382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7321c-4932-4d83-846c-12ba1c15d3e0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.756428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd4519-5c0e-4e2a-825c-2c883dbe3177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.757094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd4519-5c0e-4e2a-825c-2c883dbe3177 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.757381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd4519-5c0e-4e2a-825c-2c883dbe3177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.757652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bd4519-5c0e-4e2a-825c-2c883dbe3177 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.759934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd4519-5c0e-4e2a-825c-2c883dbe3177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.763859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bd4519-5c0e-4e2a-825c-2c883dbe3177 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.782308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:01.815171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 912/1814] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:01 2022] POST /identity/v3/users => generated 292 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:01.818800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd4519-5c0e-4e2a-825c-2c883dbe3177 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.838956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bd4519-5c0e-4e2a-825c-2c883dbe3177 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.850809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29380b01-39d4-4a2c-938a-0884078c77c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.851829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29380b01-39d4-4a2c-938a-0884078c77c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.852256 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29380b01-39d4-4a2c-938a-0884078c77c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.853819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29380b01-39d4-4a2c-938a-0884078c77c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.858675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 903/1815] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:01.858919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29380b01-39d4-4a2c-938a-0884078c77c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.858919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29380b01-39d4-4a2c-938a-0884078c77c0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.874087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea0a16-227f-479f-bb11-89bed6f51dd6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.875680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2eea0a16-227f-479f-bb11-89bed6f51dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:01.880085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 913/1816] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:01.882313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29380b01-39d4-4a2c-938a-0884078c77c0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.901511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29380b01-39d4-4a2c-938a-0884078c77c0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.910291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df591039-4ae5-4ed5-862a-e4ec5e15fd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.910958 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df591039-4ae5-4ed5-862a-e4ec5e15fd4e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.911246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df591039-4ae5-4ed5-862a-e4ec5e15fd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.911528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df591039-4ae5-4ed5-862a-e4ec5e15fd4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.912270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df591039-4ae5-4ed5-862a-e4ec5e15fd4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.913980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df591039-4ae5-4ed5-862a-e4ec5e15fd4e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.935255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 914/1817] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:01.939995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df591039-4ae5-4ed5-862a-e4ec5e15fd4e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:01.953707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df591039-4ae5-4ed5-862a-e4ec5e15fd4e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.959898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4063fac-bf19-479f-82a5-02d7e43d86e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.960794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4063fac-bf19-479f-82a5-02d7e43d86e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.961115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4063fac-bf19-479f-82a5-02d7e43d86e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.961370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4063fac-bf19-479f-82a5-02d7e43d86e8 admin admin] PATH_INFO: `/v3/projects/0e91937c3df949278940f93fc8c68f8b/users/58eab356e3154fe58201c0d2c10e7de9/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:01.983944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4063fac-bf19-479f-82a5-02d7e43d86e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e91937c3df949278940f93fc8c68f8b, user_id=58eab356e3154fe58201c0d2c10e7de9, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:01.987022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea0a16-227f-479f-bb11-89bed6f51dd6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:01.988447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4063fac-bf19-479f-82a5-02d7e43d86e8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:01.994758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d0b6f-e85c-452e-b683-5117dcd7cf21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:01.995463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d0b6f-e85c-452e-b683-5117dcd7cf21 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:01.995998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d0b6f-e85c-452e-b683-5117dcd7cf21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:01.996187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1d0b6f-e85c-452e-b683-5117dcd7cf21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:02.009730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d0b6f-e85c-452e-b683-5117dcd7cf21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:02.027907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1d0b6f-e85c-452e-b683-5117dcd7cf21 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:02.028894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 915/1818] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:01 2022] PUT /identity/v3/projects/0e91937c3df949278940f93fc8c68f8b/users/58eab356e3154fe58201c0d2c10e7de9/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:02.044987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6b03b-e460-4676-8ce6-d976695bbbfe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:02.045597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6b03b-e460-4676-8ce6-d976695bbbfe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:02.046213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6b03b-e460-4676-8ce6-d976695bbbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:02.066698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 904/1819] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:02.089150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2cf6b03b-e460-4676-8ce6-d976695bbbfe None None] MFA Rules not processed for user `58eab356e3154fe58201c0d2c10e7de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:02.091443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2cf6b03b-e460-4676-8ce6-d976695bbbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:02.148889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:02.195863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d0b6f-e85c-452e-b683-5117dcd7cf21 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:02.203299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 916/1820] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:02.204392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6b03b-e460-4676-8ce6-d976695bbbfe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:02.214382 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1d0b6f-e85c-452e-b683-5117dcd7cf21 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:02.288596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7c7b4-0c2b-4343-98e0-b687efd207c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:02.289447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7c7b4-0c2b-4343-98e0-b687efd207c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:02.289900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7c7b4-0c2b-4343-98e0-b687efd207c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:02.290427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc7c7b4-0c2b-4343-98e0-b687efd207c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:02.293625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4fc7c7b4-0c2b-4343-98e0-b687efd207c3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:02.300104 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6b03b-e460-4676-8ce6-d976695bbbfe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:02.307579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4f07-afa0-46d8-b62a-0a4c19184ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:02.308239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4f07-afa0-46d8-b62a-0a4c19184ccc admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:02.308522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4f07-afa0-46d8-b62a-0a4c19184ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:02.308798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e4f07-afa0-46d8-b62a-0a4c19184ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:02.309804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e4f07-afa0-46d8-b62a-0a4c19184ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:02.312785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812e4f07-afa0-46d8-b62a-0a4c19184ccc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:02.359817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 917/1821] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:02 2022] POST /identity/v3/projects => generated 387 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:02.360859 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4f07-afa0-46d8-b62a-0a4c19184ccc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:02.407756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7c7b4-0c2b-4343-98e0-b687efd207c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:02.409433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc7c7b4-0c2b-4343-98e0-b687efd207c3 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:02.439098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 905/1822] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:02 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:02.509019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e4f07-afa0-46d8-b62a-0a4c19184ccc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:02.521913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd748-5c06-4c7b-b5bd-de5c37dc70ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:02.522523 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd748-5c06-4c7b-b5bd-de5c37dc70ec admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:02.522861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd748-5c06-4c7b-b5bd-de5c37dc70ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:02.523131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd748-5c06-4c7b-b5bd-de5c37dc70ec admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:02.523834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcd748-5c06-4c7b-b5bd-de5c37dc70ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:02.526020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcd748-5c06-4c7b-b5bd-de5c37dc70ec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:02.563013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 918/1823] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:02 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:02.567058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7c7b4-0c2b-4343-98e0-b687efd207c3 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:02.580685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc7c7b4-0c2b-4343-98e0-b687efd207c3 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:02.591759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ac2b9-7e8c-402e-9acc-6c5c9d96beef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:02.592486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ac2b9-7e8c-402e-9acc-6c5c9d96beef admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:02.592773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ac2b9-7e8c-402e-9acc-6c5c9d96beef admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:02.593051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ac2b9-7e8c-402e-9acc-6c5c9d96beef admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:02.593783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ac2b9-7e8c-402e-9acc-6c5c9d96beef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:02.595780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ac2b9-7e8c-402e-9acc-6c5c9d96beef admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:02.622511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 906/1824] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:02.643173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd748-5c06-4c7b-b5bd-de5c37dc70ec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:02.661445 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd748-5c06-4c7b-b5bd-de5c37dc70ec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:02.667862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57798733-f681-4f0a-872a-ad82f353e9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:02.668724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57798733-f681-4f0a-872a-ad82f353e9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:02.669008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57798733-f681-4f0a-872a-ad82f353e9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:02.669482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57798733-f681-4f0a-872a-ad82f353e9ca admin admin] PATH_INFO: `/v3/projects/63116211a5074e94992a3b31e45f2d92/users/8f434f81209f4faebc738a3f8b50fed3/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:02.700720 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57798733-f681-4f0a-872a-ad82f353e9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=63116211a5074e94992a3b31e45f2d92, user_id=8f434f81209f4faebc738a3f8b50fed3, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:02.704668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57798733-f681-4f0a-872a-ad82f353e9ca admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:02.734113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ac2b9-7e8c-402e-9acc-6c5c9d96beef admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:02.747007 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 919/1825] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:02 2022] PUT /identity/v3/projects/63116211a5074e94992a3b31e45f2d92/users/8f434f81209f4faebc738a3f8b50fed3/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:02.749786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ac2b9-7e8c-402e-9acc-6c5c9d96beef admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:02.794009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:02.930036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4b417-01c4-49c8-90ed-7097866d19c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:02.931766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4b417-01c4-49c8-90ed-7097866d19c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:02.932055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4b417-01c4-49c8-90ed-7097866d19c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:02.932333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4b417-01c4-49c8-90ed-7097866d19c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:02.937337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4b417-01c4-49c8-90ed-7097866d19c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:02.941020 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4b417-01c4-49c8-90ed-7097866d19c7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:02.978562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 907/1826] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:02.981028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57798733-f681-4f0a-872a-ad82f353e9ca admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:02.995855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57798733-f681-4f0a-872a-ad82f353e9ca admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:03.004350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abda4b-4152-4c35-9bdb-2f147f970f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:03.005102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abda4b-4152-4c35-9bdb-2f147f970f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:03.005411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abda4b-4152-4c35-9bdb-2f147f970f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:03.005755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abda4b-4152-4c35-9bdb-2f147f970f89 admin admin] PATH_INFO: `/v3/projects/63116211a5074e94992a3b31e45f2d92/users/8f434f81209f4faebc738a3f8b50fed3/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:03.007930 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abda4b-4152-4c35-9bdb-2f147f970f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63116211a5074e94992a3b31e45f2d92, user_id=8f434f81209f4faebc738a3f8b50fed3, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:03.011963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abda4b-4152-4c35-9bdb-2f147f970f89 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:03.076080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 920/1827] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:02 2022] PUT /identity/v3/projects/63116211a5074e94992a3b31e45f2d92/users/8f434f81209f4faebc738a3f8b50fed3/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:03.078587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19961-3984-4f03-8924-f6f30a7f0bab None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:03.079047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19961-3984-4f03-8924-f6f30a7f0bab None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:03.079406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e19961-3984-4f03-8924-f6f30a7f0bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:03.207875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b4e19961-3984-4f03-8924-f6f30a7f0bab None None] MFA Rules not processed for user `8f434f81209f4faebc738a3f8b50fed3`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:03.208050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abda4b-4152-4c35-9bdb-2f147f970f89 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:03.209096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4e19961-3984-4f03-8924-f6f30a7f0bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:03.210349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e1abda4b-4152-4c35-9bdb-2f147f970f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:03.265559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:03.396994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 908/1828] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:03.410239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abda4b-4152-4c35-9bdb-2f147f970f89 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:03.410702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e19961-3984-4f03-8924-f6f30a7f0bab None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:03.412468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b4e19961-3984-4f03-8924-f6f30a7f0bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:03.432830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:03.443484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:03.444156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:03.444657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] PATH_INFO: `/v3/users/555b58a0a8a04a60a25c96c19d956ab6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:03.447809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=555b58a0a8a04a60a25c96c19d956ab6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:03.451423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:03.621389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555b58a0a8a04a60a25c96c19d956ab6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:03.632894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555b58a0a8a04a60a25c96c19d956ab6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:03.657315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e19961-3984-4f03-8924-f6f30a7f0bab None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:03.666329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2de75-d6bd-46d6-a977-56e5b69b1416 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:03.669327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2de75-d6bd-46d6-a977-56e5b69b1416 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:03.669653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2de75-d6bd-46d6-a977-56e5b69b1416 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:03.669979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a2de75-d6bd-46d6-a977-56e5b69b1416 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:03.675281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555b58a0a8a04a60a25c96c19d956ab6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:03.700192 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555b58a0a8a04a60a25c96c19d956ab6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:03.710911 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2de75-d6bd-46d6-a977-56e5b69b1416 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:03.716489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b2daef0b-2624-4439-8cf0-8bca5caa3ce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555b58a0a8a04a60a25c96c19d956ab6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:03.720048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a2de75-d6bd-46d6-a977-56e5b69b1416 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:03.727914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 921/1829] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:03 2022] DELETE /identity/v3/users/555b58a0a8a04a60a25c96c19d956ab6 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:03.732171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0af0bc-dd1c-4738-8bbc-9216170b9d9b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:03.732855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0af0bc-dd1c-4738-8bbc-9216170b9d9b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:03.733339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0af0bc-dd1c-4738-8bbc-9216170b9d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:03.782476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 909/1830] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:03.787993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0a0af0bc-dd1c-4738-8bbc-9216170b9d9b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:03.800806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a0af0bc-dd1c-4738-8bbc-9216170b9d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:03.858170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2de75-d6bd-46d6-a977-56e5b69b1416 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:03.878453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a2de75-d6bd-46d6-a977-56e5b69b1416 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:03.965251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 922/1831] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:04.042740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:04.074851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0af0bc-dd1c-4738-8bbc-9216170b9d9b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:04.077528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a0af0bc-dd1c-4738-8bbc-9216170b9d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:04.103461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:04.104413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:04.104828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:04.105201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] PATH_INFO: `/v3/users/58eab356e3154fe58201c0d2c10e7de9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:04.110653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58eab356e3154fe58201c0d2c10e7de9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:04.112794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:04.163160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:04.223799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0af0bc-dd1c-4738-8bbc-9216170b9d9b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:04.232843 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba57a580-c262-4391-b13e-9b10e23b9983 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:04.235870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57a580-c262-4391-b13e-9b10e23b9983 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:04.237035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57a580-c262-4391-b13e-9b10e23b9983 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:04.237383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba57a580-c262-4391-b13e-9b10e23b9983 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:04.249731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58eab356e3154fe58201c0d2c10e7de9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:04.251279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58eab356e3154fe58201c0d2c10e7de9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:04.259712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57a580-c262-4391-b13e-9b10e23b9983 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:04.264951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba57a580-c262-4391-b13e-9b10e23b9983 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:04.285529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58eab356e3154fe58201c0d2c10e7de9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:04.291975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 923/1832] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:04.296807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58eab356e3154fe58201c0d2c10e7de9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:04.313319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58eab356e3154fe58201c0d2c10e7de9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:04.325006 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 910/1833] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:03 2022] DELETE /identity/v3/users/58eab356e3154fe58201c0d2c10e7de9 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:04.327290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3bacb-8ee7-4264-9a93-0dcb314faeda None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:04.327681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3bacb-8ee7-4264-9a93-0dcb314faeda None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:04.328047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a3bacb-8ee7-4264-9a93-0dcb314faeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:04.365330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-03a3bacb-8ee7-4264-9a93-0dcb314faeda None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:04.366860 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-03a3bacb-8ee7-4264-9a93-0dcb314faeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:04.457904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 924/1834] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:04.480062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:04.482097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:04.634836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9699ae-4483-4eac-8802-ac0a3a5f72c4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:04.645042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb723468-565a-4470-a437-dfabfc6f1b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:04.646133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb723468-565a-4470-a437-dfabfc6f1b0a service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:04.646698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb723468-565a-4470-a437-dfabfc6f1b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:04.647671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb723468-565a-4470-a437-dfabfc6f1b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:04.662502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb723468-565a-4470-a437-dfabfc6f1b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:04.665436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb723468-565a-4470-a437-dfabfc6f1b0a service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:04.683973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 911/1835] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:05.416615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3bacb-8ee7-4264-9a93-0dcb314faeda None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:05.418115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-03a3bacb-8ee7-4264-9a93-0dcb314faeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:05.440994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a3bacb-8ee7-4264-9a93-0dcb314faeda None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:05.455628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:05.456705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:05.457503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:05.458515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] PATH_INFO: `/v3/projects/f8a9a7e2d28e4f56855abfcb939eccee` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:05.460605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8a9a7e2d28e4f56855abfcb939eccee)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:05.462934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:05.600954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8a9a7e2d28e4f56855abfcb939eccee', 'request_id': 'req-cb2a2745-0155-4126-ad07-be37edbf0c81'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:05.608570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8a9a7e2d28e4f56855abfcb939eccee', 'request_id': 'req-cb2a2745-0155-4126-ad07-be37edbf0c81'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:05.615393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8a9a7e2d28e4f56855abfcb939eccee', 'request_id': 'req-cb2a2745-0155-4126-ad07-be37edbf0c81'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:05.646047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] The token cache is being invalidate because project f8a9a7e2d28e4f56855abfcb939eccee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:05.646439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:05.653906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb2a2745-0155-4126-ad07-be37edbf0c81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:05.656434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 925/1836] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:05 2022] DELETE /identity/v3/projects/f8a9a7e2d28e4f56855abfcb939eccee => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:05.660400 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb723468-565a-4470-a437-dfabfc6f1b0a service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:05.674247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb723468-565a-4470-a437-dfabfc6f1b0a service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:05.691199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead9fe5-19e7-4529-87fa-9e09bd4ab061 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:05.691665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead9fe5-19e7-4529-87fa-9e09bd4ab061 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:05.691754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead9fe5-19e7-4529-87fa-9e09bd4ab061 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:05.693388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 926/1837] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:26:05.694751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead9fe5-19e7-4529-87fa-9e09bd4ab061 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:05.697034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eead9fe5-19e7-4529-87fa-9e09bd4ab061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:05.876460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead9fe5-19e7-4529-87fa-9e09bd4ab061 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:05.900328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:05.901134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:05.901513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:05.902001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] PATH_INFO: `/v3/projects/0e91937c3df949278940f93fc8c68f8b` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:05.903378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e91937c3df949278940f93fc8c68f8b)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:05.905930 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:05.911810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:05.912478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:05.920416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:05.920842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:05.921241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] PATH_INFO: `/v3/users/3f3e204d9fb04cb4888d7e56bfd2b0e7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:05.923739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f3e204d9fb04cb4888d7e56bfd2b0e7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:05.932568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:05.976314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:06.063432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e91937c3df949278940f93fc8c68f8b', 'request_id': 'req-e2266d4a-6355-44b0-acf5-cb2083c288a4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.090439 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f3e204d9fb04cb4888d7e56bfd2b0e7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.091479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f3e204d9fb04cb4888d7e56bfd2b0e7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.094705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e91937c3df949278940f93fc8c68f8b', 'request_id': 'req-e2266d4a-6355-44b0-acf5-cb2083c288a4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.111976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e91937c3df949278940f93fc8c68f8b', 'request_id': 'req-e2266d4a-6355-44b0-acf5-cb2083c288a4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.128049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f3e204d9fb04cb4888d7e56bfd2b0e7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.140478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f3e204d9fb04cb4888d7e56bfd2b0e7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.144761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] The token cache is being invalidate because project 0e91937c3df949278940f93fc8c68f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:06.145057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.146717 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e2266d4a-6355-44b0-acf5-cb2083c288a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.152952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 927/1838] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:05 2022] DELETE /identity/v3/projects/0e91937c3df949278940f93fc8c68f8b => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:06.152952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ac1f2-6937-48d0-a593-7396e6916547 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:06.153451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f3e204d9fb04cb4888d7e56bfd2b0e7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.154153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ac1f2-6937-48d0-a593-7396e6916547 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:06.155991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ac1f2-6937-48d0-a593-7396e6916547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:06.177231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 912/1839] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:05 2022] DELETE /identity/v3/users/3f3e204d9fb04cb4888d7e56bfd2b0e7 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:06.178301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:06.179963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:06.214568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-656ac1f2-6937-48d0-a593-7396e6916547 None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:06.219891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-656ac1f2-6937-48d0-a593-7396e6916547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:06.334341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7d0ba-272d-4cac-9643-7784230190cb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:06.348722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:06.349534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:06.349693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:06.350157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] PATH_INFO: `/v3/users/8f434f81209f4faebc738a3f8b50fed3` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:06.352959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f434f81209f4faebc738a3f8b50fed3)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:06.357640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:06.393633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 928/1840] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:26:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:06.415866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ac1f2-6937-48d0-a593-7396e6916547 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:06.420733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-656ac1f2-6937-48d0-a593-7396e6916547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:06.445260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ac1f2-6937-48d0-a593-7396e6916547 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:06.472894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6a281-7cd3-49e2-93f7-2057e96a563e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:06.474988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6a281-7cd3-49e2-93f7-2057e96a563e service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:06.476207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6a281-7cd3-49e2-93f7-2057e96a563e service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:06.476555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6a281-7cd3-49e2-93f7-2057e96a563e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:06.489498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6a281-7cd3-49e2-93f7-2057e96a563e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:06.491526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6a281-7cd3-49e2-93f7-2057e96a563e service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:06.492361 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f434f81209f4faebc738a3f8b50fed3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.493107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f434f81209f4faebc738a3f8b50fed3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.515765 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f434f81209f4faebc738a3f8b50fed3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.522692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 929/1841] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:06.533064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f434f81209f4faebc738a3f8b50fed3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.543840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-de4676bb-50b3-4bfa-a672-8013b54aff63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f434f81209f4faebc738a3f8b50fed3'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.553812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 913/1842] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:06 2022] DELETE /identity/v3/users/8f434f81209f4faebc738a3f8b50fed3 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:06.640142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6a281-7cd3-49e2-93f7-2057e96a563e service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:06.641769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dbe6a281-7cd3-49e2-93f7-2057e96a563e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:06.672622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6a281-7cd3-49e2-93f7-2057e96a563e service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:06.759655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:06.760651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:06.760977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:06.761353 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] PATH_INFO: `/v3/projects/702974231ce34b2d9cdc5dc48d4da831` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:06.762720 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] RBAC: Authorizing `identity:delete_project(project_id=702974231ce34b2d9cdc5dc48d4da831)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:06.766010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:06.780736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:06.837230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '702974231ce34b2d9cdc5dc48d4da831', 'request_id': 'req-3f61ff19-0f23-43a0-835b-b928f3b43612'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.847806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '702974231ce34b2d9cdc5dc48d4da831', 'request_id': 'req-3f61ff19-0f23-43a0-835b-b928f3b43612'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.857261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '702974231ce34b2d9cdc5dc48d4da831', 'request_id': 'req-3f61ff19-0f23-43a0-835b-b928f3b43612'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.890010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] The token cache is being invalidate because project 702974231ce34b2d9cdc5dc48d4da831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:06.890295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.890994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:06.895982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 930/1843] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:06 2022] DELETE /identity/v3/projects/702974231ce34b2d9cdc5dc48d4da831 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:06.898295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7c46f-8e5e-454e-9d3e-d38a65d8c937 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:06.898754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7c46f-8e5e-454e-9d3e-d38a65d8c937 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:06.898954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa7c46f-8e5e-454e-9d3e-d38a65d8c937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:06.946331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4fa7c46f-8e5e-454e-9d3e-d38a65d8c937 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:06.949203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4fa7c46f-8e5e-454e-9d3e-d38a65d8c937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:06.956863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:07.072974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.075015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:07.089511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 914/1844] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:07.096413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb3157-f6eb-40bd-8ede-c9b20c8ecfe3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.096712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb3157-f6eb-40bd-8ede-c9b20c8ecfe3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.097051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb3157-f6eb-40bd-8ede-c9b20c8ecfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.099141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61ff19-0f23-43a0-835b-b928f3b43612 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:07.113931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:07.114709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.115088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.115457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] PATH_INFO: `/v3/projects/63116211a5074e94992a3b31e45f2d92` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.119662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=63116211a5074e94992a3b31e45f2d92)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:07.122291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:07.169198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-28cb3157-f6eb-40bd-8ede-c9b20c8ecfe3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:07.170811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28cb3157-f6eb-40bd-8ede-c9b20c8ecfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:07.218460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 915/1845] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:07.222472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb3157-f6eb-40bd-8ede-c9b20c8ecfe3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.249898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63116211a5074e94992a3b31e45f2d92', 'request_id': 'req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:07.253835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb3157-f6eb-40bd-8ede-c9b20c8ecfe3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:07.258396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63116211a5074e94992a3b31e45f2d92', 'request_id': 'req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:07.266973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63116211a5074e94992a3b31e45f2d92', 'request_id': 'req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:07.287371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] The token cache is being invalidate because project 63116211a5074e94992a3b31e45f2d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:07.288180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:07.289052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e17ee5d3-03d7-4146-b86a-04a8fa60eebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:07.295374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 931/1846] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:07 2022] DELETE /identity/v3/projects/63116211a5074e94992a3b31e45f2d92 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:07.299986 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b0c0c-737b-4eae-b8ba-b7aa6b1407f6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.300553 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b0c0c-737b-4eae-b8ba-b7aa6b1407f6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.301049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b0c0c-737b-4eae-b8ba-b7aa6b1407f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.328151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b98b0c0c-737b-4eae-b8ba-b7aa6b1407f6 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:07.330244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b98b0c0c-737b-4eae-b8ba-b7aa6b1407f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:07.395780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfec32-2107-4638-9981-00b76e70e9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:07.400397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfec32-2107-4638-9981-00b76e70e9fe admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.400726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfec32-2107-4638-9981-00b76e70e9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.403113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecfec32-2107-4638-9981-00b76e70e9fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.404546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfec32-2107-4638-9981-00b76e70e9fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:07.404865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfec32-2107-4638-9981-00b76e70e9fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:07.414490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecfec32-2107-4638-9981-00b76e70e9fe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:07.427116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 932/1847] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:07.435140 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea79365-bddb-468d-8041-027e9b31f480 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.435594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea79365-bddb-468d-8041-027e9b31f480 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.435974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea79365-bddb-468d-8041-027e9b31f480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.446279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 916/1848] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:07.447580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfec32-2107-4638-9981-00b76e70e9fe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.451319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-eecfec32-2107-4638-9981-00b76e70e9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:07.473927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecfec32-2107-4638-9981-00b76e70e9fe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:07.478523 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9ea79365-bddb-468d-8041-027e9b31f480 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:07.482537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b1236-2d8e-4c60-a9a8-b25874016ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:07.483631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b1236-2d8e-4c60-a9a8-b25874016ead admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.484899 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b1236-2d8e-4c60-a9a8-b25874016ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.485278 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b1236-2d8e-4c60-a9a8-b25874016ead admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.486105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b1236-2d8e-4c60-a9a8-b25874016ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:07.489601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b1236-2d8e-4c60-a9a8-b25874016ead admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:07.495006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9ea79365-bddb-468d-8041-027e9b31f480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:07.526579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 933/1849] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:07.528726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea79365-bddb-468d-8041-027e9b31f480 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.534249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 917/1850] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:07 2022] POST /identity/v3/projects => generated 383 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:07.539915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b1236-2d8e-4c60-a9a8-b25874016ead admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.551928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea79365-bddb-468d-8041-027e9b31f480 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:07.582159 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:07.658818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea621ce-2c4a-4093-b78f-56c4f6cb023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:07.659297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea621ce-2c4a-4093-b78f-56c4f6cb023b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.659579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea621ce-2c4a-4093-b78f-56c4f6cb023b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.659850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea621ce-2c4a-4093-b78f-56c4f6cb023b admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.660559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea621ce-2c4a-4093-b78f-56c4f6cb023b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:07.660862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea621ce-2c4a-4093-b78f-56c4f6cb023b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:07.662558 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea621ce-2c4a-4093-b78f-56c4f6cb023b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:07.668712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b1236-2d8e-4c60-a9a8-b25874016ead admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:07.682489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5d44b-9df6-4eb2-9e70-669f5f114519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:07.683207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5d44b-9df6-4eb2-9e70-669f5f114519 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.683494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5d44b-9df6-4eb2-9e70-669f5f114519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.683757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5d44b-9df6-4eb2-9e70-669f5f114519 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.684441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5d44b-9df6-4eb2-9e70-669f5f114519 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:07.686480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5d44b-9df6-4eb2-9e70-669f5f114519 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:07.703748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 934/1851] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:07.705147 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea621ce-2c4a-4093-b78f-56c4f6cb023b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.738666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea621ce-2c4a-4093-b78f-56c4f6cb023b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:07.742538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 918/1852] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:07 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:07.746889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5d44b-9df6-4eb2-9e70-669f5f114519 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.751879 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd2d1-2cf5-44ef-9181-f9bde4ca7b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:07.752588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd2d1-2cf5-44ef-9181-f9bde4ca7b87 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.752868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd2d1-2cf5-44ef-9181-f9bde4ca7b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.753141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd2d1-2cf5-44ef-9181-f9bde4ca7b87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.753910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698dd2d1-2cf5-44ef-9181-f9bde4ca7b87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:07.757248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698dd2d1-2cf5-44ef-9181-f9bde4ca7b87 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:07.759110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:07.770427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5d44b-9df6-4eb2-9e70-669f5f114519 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:07.780218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f8008-d16e-4946-a01e-94fd04ea7eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:07.781819 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f8008-d16e-4946-a01e-94fd04ea7eaf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.782801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f8008-d16e-4946-a01e-94fd04ea7eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.783393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f8008-d16e-4946-a01e-94fd04ea7eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.784526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f8008-d16e-4946-a01e-94fd04ea7eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:07.787061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f8008-d16e-4946-a01e-94fd04ea7eaf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:07.814207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 919/1853] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:07.818224 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f8008-d16e-4946-a01e-94fd04ea7eaf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.824444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 935/1854] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:07 2022] POST /identity/v3/projects => generated 389 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:07.830327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd2d1-2cf5-44ef-9181-f9bde4ca7b87 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.838872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f8008-d16e-4946-a01e-94fd04ea7eaf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:07.914638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd2d1-2cf5-44ef-9181-f9bde4ca7b87 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:07.920385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5553e08-c67f-4eb0-9138-37a9a4c00c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:07.921069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5553e08-c67f-4eb0-9138-37a9a4c00c2d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.921350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5553e08-c67f-4eb0-9138-37a9a4c00c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.921626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5553e08-c67f-4eb0-9138-37a9a4c00c2d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.922681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5553e08-c67f-4eb0-9138-37a9a4c00c2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:07.924831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5553e08-c67f-4eb0-9138-37a9a4c00c2d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:07.971169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f02d3b-9f22-452e-b754-bbacb3e9c248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:07.971950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 936/1855] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:07 2022] POST /identity/v3/users => generated 295 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:07.972879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f02d3b-9f22-452e-b754-bbacb3e9c248 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.973358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f02d3b-9f22-452e-b754-bbacb3e9c248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.973857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f02d3b-9f22-452e-b754-bbacb3e9c248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.975146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f02d3b-9f22-452e-b754-bbacb3e9c248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:07.977193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f02d3b-9f22-452e-b754-bbacb3e9c248 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:07.977347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5553e08-c67f-4eb0-9138-37a9a4c00c2d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.988911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5553e08-c67f-4eb0-9138-37a9a4c00c2d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:07.992791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 920/1856] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:07.993723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f02d3b-9f22-452e-b754-bbacb3e9c248 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:07.996308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da179e0-1397-4fd8-84c1-2e6897ecb2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:07.997465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da179e0-1397-4fd8-84c1-2e6897ecb2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:07.998006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da179e0-1397-4fd8-84c1-2e6897ecb2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:07.998439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da179e0-1397-4fd8-84c1-2e6897ecb2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:07.999985 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da179e0-1397-4fd8-84c1-2e6897ecb2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.002388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da179e0-1397-4fd8-84c1-2e6897ecb2e6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.005325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f02d3b-9f22-452e-b754-bbacb3e9c248 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.011304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b973e3e-3c55-498c-841d-e3ed7232165b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.012028 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973e3e-3c55-498c-841d-e3ed7232165b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.012313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973e3e-3c55-498c-841d-e3ed7232165b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.012634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b973e3e-3c55-498c-841d-e3ed7232165b admin admin] PATH_INFO: `/v3/projects/df3d1d84fc6d430f8506c7578bbe61fd/users/2e8cf602969c4d83bbbf71333c39ae87/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.020268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 937/1857] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:08.021508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da179e0-1397-4fd8-84c1-2e6897ecb2e6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.032366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b973e3e-3c55-498c-841d-e3ed7232165b admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3d1d84fc6d430f8506c7578bbe61fd, user_id=2e8cf602969c4d83bbbf71333c39ae87, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.034789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b973e3e-3c55-498c-841d-e3ed7232165b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.035429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da179e0-1397-4fd8-84c1-2e6897ecb2e6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.041689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dff66-7794-4622-a845-9d67d1a1941b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.042586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dff66-7794-4622-a845-9d67d1a1941b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.043123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dff66-7794-4622-a845-9d67d1a1941b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.043524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dff66-7794-4622-a845-9d67d1a1941b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.044361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dff66-7794-4622-a845-9d67d1a1941b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.046202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dff66-7794-4622-a845-9d67d1a1941b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.073993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 938/1858] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:08.075556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dff66-7794-4622-a845-9d67d1a1941b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.091695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dff66-7794-4622-a845-9d67d1a1941b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.093992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 921/1859] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:07 2022] PUT /identity/v3/projects/df3d1d84fc6d430f8506c7578bbe61fd/users/2e8cf602969c4d83bbbf71333c39ae87/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:08.106225 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f99524-a5ed-4b1d-8070-812ef615592a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.106616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f99524-a5ed-4b1d-8070-812ef615592a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.106907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f99524-a5ed-4b1d-8070-812ef615592a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.156698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-49f99524-a5ed-4b1d-8070-812ef615592a None None] MFA Rules not processed for user `2e8cf602969c4d83bbbf71333c39ae87`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:08.158033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-49f99524-a5ed-4b1d-8070-812ef615592a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:08.170617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:08.182166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d09f2f-11bf-473e-8b58-277d115c1b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.182905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d09f2f-11bf-473e-8b58-277d115c1b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.183188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d09f2f-11bf-473e-8b58-277d115c1b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.183463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d09f2f-11bf-473e-8b58-277d115c1b43 admin admin] PATH_INFO: `/v3/projects/8e930f3c3f4549879221babe49e1cffc/users/51ca090aee9042a8b838ce4072d4f521/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.239423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d09f2f-11bf-473e-8b58-277d115c1b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e930f3c3f4549879221babe49e1cffc, user_id=51ca090aee9042a8b838ce4072d4f521, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.243349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d09f2f-11bf-473e-8b58-277d115c1b43 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.258061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 939/1860] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:08 2022] PUT /identity/v3/projects/8e930f3c3f4549879221babe49e1cffc/users/51ca090aee9042a8b838ce4072d4f521/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:08.259441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 922/1861] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:08.277988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d09f2f-11bf-473e-8b58-277d115c1b43 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.278668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771b786-7410-4ffd-88ee-533766a6910a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.278668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771b786-7410-4ffd-88ee-533766a6910a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.278668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e771b786-7410-4ffd-88ee-533766a6910a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.320885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e771b786-7410-4ffd-88ee-533766a6910a None None] MFA Rules not processed for user `51ca090aee9042a8b838ce4072d4f521`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:08.322504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e771b786-7410-4ffd-88ee-533766a6910a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:08.362596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d09f2f-11bf-473e-8b58-277d115c1b43 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.368712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3b158-2555-42d3-923a-76b8e0ecef14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.369596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3b158-2555-42d3-923a-76b8e0ecef14 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.370668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3b158-2555-42d3-923a-76b8e0ecef14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.371103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b3b158-2555-42d3-923a-76b8e0ecef14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.371884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3b158-2555-42d3-923a-76b8e0ecef14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.374334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b3b158-2555-42d3-923a-76b8e0ecef14 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.395536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:08.413451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 940/1862] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:08 2022] POST /identity/v3/projects => generated 379 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:08.420915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3b158-2555-42d3-923a-76b8e0ecef14 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.434099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 923/1863] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:08.443721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771b786-7410-4ffd-88ee-533766a6910a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.463467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e771b786-7410-4ffd-88ee-533766a6910a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.515742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b3b158-2555-42d3-923a-76b8e0ecef14 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.523822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691ae0e-7596-40e3-aabc-5c38fc1a1ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.524695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691ae0e-7596-40e3-aabc-5c38fc1a1ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.525870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691ae0e-7596-40e3-aabc-5c38fc1a1ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.526485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691ae0e-7596-40e3-aabc-5c38fc1a1ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.527338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691ae0e-7596-40e3-aabc-5c38fc1a1ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.530824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691ae0e-7596-40e3-aabc-5c38fc1a1ca6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.569810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 941/1864] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:08 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:08.573655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691ae0e-7596-40e3-aabc-5c38fc1a1ca6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.588061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691ae0e-7596-40e3-aabc-5c38fc1a1ca6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.589139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783a987-bb3f-44b2-8ceb-ccffea0fda5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.589826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783a987-bb3f-44b2-8ceb-ccffea0fda5a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.590464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783a987-bb3f-44b2-8ceb-ccffea0fda5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.590464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783a987-bb3f-44b2-8ceb-ccffea0fda5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.591519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783a987-bb3f-44b2-8ceb-ccffea0fda5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.595202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82363bbc-a40d-4b31-acd9-ce691599995a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.596018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82363bbc-a40d-4b31-acd9-ce691599995a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.596369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82363bbc-a40d-4b31-acd9-ce691599995a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.596880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82363bbc-a40d-4b31-acd9-ce691599995a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.597376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783a987-bb3f-44b2-8ceb-ccffea0fda5a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.598077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82363bbc-a40d-4b31-acd9-ce691599995a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.600664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82363bbc-a40d-4b31-acd9-ce691599995a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.621143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 942/1865] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:08.623947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82363bbc-a40d-4b31-acd9-ce691599995a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.645595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82363bbc-a40d-4b31-acd9-ce691599995a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.654928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5ffec-fe5e-451e-a0a2-85031c930685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.656638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5ffec-fe5e-451e-a0a2-85031c930685 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.657084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5ffec-fe5e-451e-a0a2-85031c930685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.657474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf5ffec-fe5e-451e-a0a2-85031c930685 admin admin] PATH_INFO: `/v3/projects/b6165a1dd1164cf0856dfc34946311f6/users/e187533e8a39464e814b542e0eaabb81/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.663184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 924/1866] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:08 2022] POST /identity/v3/projects => generated 387 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:08.667503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783a987-bb3f-44b2-8ceb-ccffea0fda5a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.684820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5ffec-fe5e-451e-a0a2-85031c930685 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6165a1dd1164cf0856dfc34946311f6, user_id=e187533e8a39464e814b542e0eaabb81, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.690405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf5ffec-fe5e-451e-a0a2-85031c930685 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.716347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 943/1867] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:08 2022] PUT /identity/v3/projects/b6165a1dd1164cf0856dfc34946311f6/users/e187533e8a39464e814b542e0eaabb81/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:08.727094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5ffec-fe5e-451e-a0a2-85031c930685 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.758358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:08.825873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf5ffec-fe5e-451e-a0a2-85031c930685 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.834135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de2aa9-bbff-4dcb-a5cc-f7ba87de985d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.834796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de2aa9-bbff-4dcb-a5cc-f7ba87de985d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.835251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de2aa9-bbff-4dcb-a5cc-f7ba87de985d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.835590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de2aa9-bbff-4dcb-a5cc-f7ba87de985d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.837008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de2aa9-bbff-4dcb-a5cc-f7ba87de985d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.840023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de2aa9-bbff-4dcb-a5cc-f7ba87de985d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.853895 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 944/1868] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:08.854241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783a987-bb3f-44b2-8ceb-ccffea0fda5a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.858451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de2aa9-bbff-4dcb-a5cc-f7ba87de985d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.859766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bbc78-aa19-463f-8ce9-efdde49f4584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.860611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bbc78-aa19-463f-8ce9-efdde49f4584 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.860897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bbc78-aa19-463f-8ce9-efdde49f4584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.861183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bbc78-aa19-463f-8ce9-efdde49f4584 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.862227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bbc78-aa19-463f-8ce9-efdde49f4584 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.864277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9bbc78-aa19-463f-8ce9-efdde49f4584 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.878901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de2aa9-bbff-4dcb-a5cc-f7ba87de985d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.890335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744d546-93a7-4616-ae72-deed85704457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:08.891849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744d546-93a7-4616-ae72-deed85704457 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:08.892604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744d546-93a7-4616-ae72-deed85704457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:08.893077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744d546-93a7-4616-ae72-deed85704457 admin admin] PATH_INFO: `/v3/projects/b6165a1dd1164cf0856dfc34946311f6/users/e187533e8a39464e814b542e0eaabb81/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:08.895535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744d546-93a7-4616-ae72-deed85704457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6165a1dd1164cf0856dfc34946311f6, user_id=e187533e8a39464e814b542e0eaabb81, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:08.899087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 925/1869] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:08 2022] POST /identity/v3/users => generated 300 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:08.899087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bbc78-aa19-463f-8ce9-efdde49f4584 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:08.899087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bd9bbc78-aa19-463f-8ce9-efdde49f4584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:08.902015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744d546-93a7-4616-ae72-deed85704457 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:08.915684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9bbc78-aa19-463f-8ce9-efdde49f4584 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:08.921313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 945/1870] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:08 2022] PUT /identity/v3/projects/b6165a1dd1164cf0856dfc34946311f6/users/e187533e8a39464e814b542e0eaabb81/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:08.922467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744d546-93a7-4616-ae72-deed85704457 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:09.005005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:09.005829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.006111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.006386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] PATH_INFO: `/v3/users/34d6259eb15e4dc4b9b6b4217aff41c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.007632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=34d6259eb15e4dc4b9b6b4217aff41c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:09.008125 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744d546-93a7-4616-ae72-deed85704457 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:09.009623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:09.011514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:09.014004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd40d75-91d1-4bd8-9ba9-4d3a117f580f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:09.014925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd40d75-91d1-4bd8-9ba9-4d3a117f580f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.015304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd40d75-91d1-4bd8-9ba9-4d3a117f580f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.015804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd40d75-91d1-4bd8-9ba9-4d3a117f580f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.017392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd40d75-91d1-4bd8-9ba9-4d3a117f580f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:09.019486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd40d75-91d1-4bd8-9ba9-4d3a117f580f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:09.032772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 946/1871] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:09.034222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79159c-7af9-4d68-9b86-de65d1d7d1f9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.034557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79159c-7af9-4d68-9b86-de65d1d7d1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.034950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79159c-7af9-4d68-9b86-de65d1d7d1f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.067369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6e79159c-7af9-4d68-9b86-de65d1d7d1f9 None None] MFA Rules not processed for user `e187533e8a39464e814b542e0eaabb81`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:09.068797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6e79159c-7af9-4d68-9b86-de65d1d7d1f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:09.070955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34d6259eb15e4dc4b9b6b4217aff41c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:09.071105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34d6259eb15e4dc4b9b6b4217aff41c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:09.087250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34d6259eb15e4dc4b9b6b4217aff41c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:09.095429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34d6259eb15e4dc4b9b6b4217aff41c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:09.106092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34d6259eb15e4dc4b9b6b4217aff41c6'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:09.117046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 926/1872] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:08 2022] DELETE /identity/v3/users/34d6259eb15e4dc4b9b6b4217aff41c6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:09.117046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:09.117955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:09.153878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 947/1873] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:09.153878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d98963-1284-4d5a-8fb5-517de927d3ce None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.153878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d98963-1284-4d5a-8fb5-517de927d3ce None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.153878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d98963-1284-4d5a-8fb5-517de927d3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.186102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f3d98963-1284-4d5a-8fb5-517de927d3ce None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:09.188573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f3d98963-1284-4d5a-8fb5-517de927d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:09.198449 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 948/1874] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:09.200458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d98963-1284-4d5a-8fb5-517de927d3ce None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:09.202725 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f3d98963-1284-4d5a-8fb5-517de927d3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:09.212886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b169b7-8abe-4137-b3e8-f611d8c6f3ce admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:09.220054 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337c85a-ef6b-4c49-a6aa-b2110aa2d666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:09.220777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337c85a-ef6b-4c49-a6aa-b2110aa2d666 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.221079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337c85a-ef6b-4c49-a6aa-b2110aa2d666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.221361 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4337c85a-ef6b-4c49-a6aa-b2110aa2d666 admin admin] PATH_INFO: `/v3/projects/3e4d00a67f6b4f9c8ff1fa3bd98b0b9f/users/c5986b12bf854bf2a83c260efbad8b96/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.231496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:09.242213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337c85a-ef6b-4c49-a6aa-b2110aa2d666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4d00a67f6b4f9c8ff1fa3bd98b0b9f, user_id=c5986b12bf854bf2a83c260efbad8b96, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:09.244910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4337c85a-ef6b-4c49-a6aa-b2110aa2d666 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:09.263079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 927/1875] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:09 2022] PUT /identity/v3/projects/3e4d00a67f6b4f9c8ff1fa3bd98b0b9f/users/c5986b12bf854bf2a83c260efbad8b96/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:09.263733 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337c85a-ef6b-4c49-a6aa-b2110aa2d666 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:09.265377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4337c85a-ef6b-4c49-a6aa-b2110aa2d666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:09.283335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d98963-1284-4d5a-8fb5-517de927d3ce None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:09.348530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4337c85a-ef6b-4c49-a6aa-b2110aa2d666 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:09.354515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d741e-389b-44ef-b79c-efa886a5784d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:09.355324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d741e-389b-44ef-b79c-efa886a5784d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.355607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d741e-389b-44ef-b79c-efa886a5784d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.355906 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d741e-389b-44ef-b79c-efa886a5784d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.369534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436f4b3-0b02-4740-8896-3a7ff93535ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:09.370429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436f4b3-0b02-4740-8896-3a7ff93535ae service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.370814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436f4b3-0b02-4740-8896-3a7ff93535ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.371349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8436f4b3-0b02-4740-8896-3a7ff93535ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.373416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:09.465795 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d741e-389b-44ef-b79c-efa886a5784d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:09.467663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d741e-389b-44ef-b79c-efa886a5784d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:09.489271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436f4b3-0b02-4740-8896-3a7ff93535ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:09.491112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8436f4b3-0b02-4740-8896-3a7ff93535ae service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:09.501222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 928/1876] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:09.505012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d741e-389b-44ef-b79c-efa886a5784d service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:09.520528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d741e-389b-44ef-b79c-efa886a5784d service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:09.520765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 949/1877] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:09.529436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c7277-45bb-47b4-b8ce-4a111e28465d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:09.530151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c7277-45bb-47b4-b8ce-4a111e28465d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.530530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c7277-45bb-47b4-b8ce-4a111e28465d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.530920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c7277-45bb-47b4-b8ce-4a111e28465d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.531932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c7277-45bb-47b4-b8ce-4a111e28465d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:09.534158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c7277-45bb-47b4-b8ce-4a111e28465d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:09.549446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 929/1878] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:09.553798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436f4b3-0b02-4740-8896-3a7ff93535ae service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:09.569847 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8436f4b3-0b02-4740-8896-3a7ff93535ae service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:09.586579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a6354-92a4-4221-92cf-d3c728b2ba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:09.587474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a6354-92a4-4221-92cf-d3c728b2ba9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.587846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a6354-92a4-4221-92cf-d3c728b2ba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.588015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3a6354-92a4-4221-92cf-d3c728b2ba9a admin admin] PATH_INFO: `/v3/projects/3e4d00a67f6b4f9c8ff1fa3bd98b0b9f/users/c5986b12bf854bf2a83c260efbad8b96/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.592887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a6354-92a4-4221-92cf-d3c728b2ba9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4d00a67f6b4f9c8ff1fa3bd98b0b9f, user_id=c5986b12bf854bf2a83c260efbad8b96, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:09.596065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3a6354-92a4-4221-92cf-d3c728b2ba9a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:09.637435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 950/1879] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:09 2022] PUT /identity/v3/projects/3e4d00a67f6b4f9c8ff1fa3bd98b0b9f/users/c5986b12bf854bf2a83c260efbad8b96/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:09.639948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbf829-22f2-4ec5-bcb4-fd699f5daf11 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.640343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbf829-22f2-4ec5-bcb4-fd699f5daf11 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.640868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfbf829-22f2-4ec5-bcb4-fd699f5daf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.667054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a6354-92a4-4221-92cf-d3c728b2ba9a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:09.717781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6cfbf829-22f2-4ec5-bcb4-fd699f5daf11 None None] MFA Rules not processed for user `c5986b12bf854bf2a83c260efbad8b96`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:09.721448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6cfbf829-22f2-4ec5-bcb4-fd699f5daf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:09.747735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3a6354-92a4-4221-92cf-d3c728b2ba9a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:09.764197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7bb35-f252-48c3-afb2-e844e71bb822 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:09.764996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7bb35-f252-48c3-afb2-e844e71bb822 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:09.765248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7bb35-f252-48c3-afb2-e844e71bb822 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:09.765526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7bb35-f252-48c3-afb2-e844e71bb822 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:09.771981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2c7bb35-f252-48c3-afb2-e844e71bb822 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:09.828173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:09.889392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 930/1880] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:09 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:09.902480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7bb35-f252-48c3-afb2-e844e71bb822 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:09.904308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7bb35-f252-48c3-afb2-e844e71bb822 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:09.921809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 951/1881] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:09.990453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbf829-22f2-4ec5-bcb4-fd699f5daf11 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:09.991844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6cfbf829-22f2-4ec5-bcb4-fd699f5daf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:10.013167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfbf829-22f2-4ec5-bcb4-fd699f5daf11 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:10.018897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb154c-7adc-4619-991b-5215430ef8d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:10.019553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb154c-7adc-4619-991b-5215430ef8d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:10.020028 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb154c-7adc-4619-991b-5215430ef8d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:10.021250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb154c-7adc-4619-991b-5215430ef8d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:10.032498 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb154c-7adc-4619-991b-5215430ef8d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:10.034984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb154c-7adc-4619-991b-5215430ef8d4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:10.054292 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 931/1882] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:10.205840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7bb35-f252-48c3-afb2-e844e71bb822 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:10.207742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c2c7bb35-f252-48c3-afb2-e844e71bb822 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:10.321474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7bb35-f252-48c3-afb2-e844e71bb822 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:10.328262 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:10.328951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:10.329229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:10.329506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] PATH_INFO: `/v3/projects/c23f83ba9d50403eac2fb8bc63dcb803` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:10.330733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c23f83ba9d50403eac2fb8bc63dcb803)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:10.332667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:10.390567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c23f83ba9d50403eac2fb8bc63dcb803', 'request_id': 'req-fcb96394-3637-44bf-88ee-68808964b49f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.399874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c23f83ba9d50403eac2fb8bc63dcb803', 'request_id': 'req-fcb96394-3637-44bf-88ee-68808964b49f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.406250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c23f83ba9d50403eac2fb8bc63dcb803', 'request_id': 'req-fcb96394-3637-44bf-88ee-68808964b49f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.427106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] The token cache is being invalidate because project c23f83ba9d50403eac2fb8bc63dcb803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:10.427459 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.428294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.429940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 952/1883] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:10 2022] DELETE /identity/v3/projects/c23f83ba9d50403eac2fb8bc63dcb803 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:10.434007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb154c-7adc-4619-991b-5215430ef8d4 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:10.445778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb154c-7adc-4619-991b-5215430ef8d4 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:10.546766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:10.548544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:10.549150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:10.549863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:10.550148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:10.550426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] PATH_INFO: `/v3/users/2d3fb63b8c5c47609045769877a312b4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:10.551631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d3fb63b8c5c47609045769877a312b4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:10.553617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:10.562434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb96394-3637-44bf-88ee-68808964b49f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:10.568222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:10.568988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:10.569335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:10.572414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] PATH_INFO: `/v3/users/51ca090aee9042a8b838ce4072d4f521` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:10.572414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ca090aee9042a8b838ce4072d4f521)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:10.573363 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:10.575163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:10.624702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d3fb63b8c5c47609045769877a312b4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.625785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d3fb63b8c5c47609045769877a312b4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.632163 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ca090aee9042a8b838ce4072d4f521'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.632925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ca090aee9042a8b838ce4072d4f521'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.635401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:10.642197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d3fb63b8c5c47609045769877a312b4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.653719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ca090aee9042a8b838ce4072d4f521'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.653945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d3fb63b8c5c47609045769877a312b4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.662129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ca090aee9042a8b838ce4072d4f521'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.665329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5c04a56c-4549-4585-a233-38eff76518dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d3fb63b8c5c47609045769877a312b4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.671138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ca090aee9042a8b838ce4072d4f521'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:10.673271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 932/1884] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:10 2022] DELETE /identity/v3/users/2d3fb63b8c5c47609045769877a312b4 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:10.677987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 953/1885] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:10 2022] DELETE /identity/v3/users/51ca090aee9042a8b838ce4072d4f521 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:10.683334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace21a2-9df4-4c19-acca-e7c3d5b7936e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:10.683734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace21a2-9df4-4c19-acca-e7c3d5b7936e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:10.683884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace21a2-9df4-4c19-acca-e7c3d5b7936e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:10.703048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cace21a2-9df4-4c19-acca-e7c3d5b7936e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:10.704263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cace21a2-9df4-4c19-acca-e7c3d5b7936e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:10.773904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 933/1886] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:10.781720 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:10.784216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:10.872649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e5f016-81ee-4f31-a237-4383bf670bae admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:10.877349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e327c-0a09-4e4a-ac94-876619ebc1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:10.878711 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e327c-0a09-4e4a-ac94-876619ebc1fd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:10.879018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e327c-0a09-4e4a-ac94-876619ebc1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:10.879350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e327c-0a09-4e4a-ac94-876619ebc1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:10.892714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e327c-0a09-4e4a-ac94-876619ebc1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:10.894222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e327c-0a09-4e4a-ac94-876619ebc1fd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:10.907622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 954/1887] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:11.168887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace21a2-9df4-4c19-acca-e7c3d5b7936e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:11.170527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cace21a2-9df4-4c19-acca-e7c3d5b7936e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:11.182711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace21a2-9df4-4c19-acca-e7c3d5b7936e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:11.187378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:11.188164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:11.188333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:11.188568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] PATH_INFO: `/v3/projects/24d07ef656e64fa4bd11dadf83447555` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:11.190083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d07ef656e64fa4bd11dadf83447555)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:11.192250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:11.266443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d07ef656e64fa4bd11dadf83447555', 'request_id': 'req-9e4755ce-fe20-4411-ae1a-d383f6e8491b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:11.272065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d07ef656e64fa4bd11dadf83447555', 'request_id': 'req-9e4755ce-fe20-4411-ae1a-d383f6e8491b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:11.283472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d07ef656e64fa4bd11dadf83447555', 'request_id': 'req-9e4755ce-fe20-4411-ae1a-d383f6e8491b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:11.297776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] The token cache is being invalidate because project 24d07ef656e64fa4bd11dadf83447555 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:11.297776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:11.298480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9e4755ce-fe20-4411-ae1a-d383f6e8491b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:11.299976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 934/1888] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:11 2022] DELETE /identity/v3/projects/24d07ef656e64fa4bd11dadf83447555 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:11.304682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772b29f-3839-4655-8087-73fdc8a5df89 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:11.304971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772b29f-3839-4655-8087-73fdc8a5df89 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:11.305245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a772b29f-3839-4655-8087-73fdc8a5df89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:11.325071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a772b29f-3839-4655-8087-73fdc8a5df89 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:11.326411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a772b29f-3839-4655-8087-73fdc8a5df89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:11.392737 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 955/1889] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:11.401839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c51375-19ea-452c-b8bd-193ffaca2f10 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:11.402129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c51375-19ea-452c-b8bd-193ffaca2f10 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:11.402406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c51375-19ea-452c-b8bd-193ffaca2f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:11.427557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-28c51375-19ea-452c-b8bd-193ffaca2f10 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:11.429960 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28c51375-19ea-452c-b8bd-193ffaca2f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:11.439516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 935/1890] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:11.444700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772b29f-3839-4655-8087-73fdc8a5df89 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:11.460787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c51375-19ea-452c-b8bd-193ffaca2f10 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:11.462497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28c51375-19ea-452c-b8bd-193ffaca2f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:11.469707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a772b29f-3839-4655-8087-73fdc8a5df89 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:11.480875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ec84-2ac1-4856-8a2d-500dde8a665a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:11.481569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ec84-2ac1-4856-8a2d-500dde8a665a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:11.484047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ec84-2ac1-4856-8a2d-500dde8a665a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:11.484383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d45ec84-2ac1-4856-8a2d-500dde8a665a admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:11.492658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45ec84-2ac1-4856-8a2d-500dde8a665a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:11.493183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45ec84-2ac1-4856-8a2d-500dde8a665a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:11.504663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d45ec84-2ac1-4856-8a2d-500dde8a665a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:11.509005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c51375-19ea-452c-b8bd-193ffaca2f10 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:11.514251 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:11.514986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:11.521795 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:11.521795 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] PATH_INFO: `/v3/projects/8e930f3c3f4549879221babe49e1cffc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:11.525777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 956/1891] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:11.527601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e930f3c3f4549879221babe49e1cffc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:11.530328 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:11.532927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ec84-2ac1-4856-8a2d-500dde8a665a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:11.546217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:11.563613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d45ec84-2ac1-4856-8a2d-500dde8a665a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:11.579253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f39391-3958-4e7c-8631-08fb95813160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:11.580175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f39391-3958-4e7c-8631-08fb95813160 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:11.580671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f39391-3958-4e7c-8631-08fb95813160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:11.581150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f39391-3958-4e7c-8631-08fb95813160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:11.583442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f39391-3958-4e7c-8631-08fb95813160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:11.593180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f39391-3958-4e7c-8631-08fb95813160 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:11.668753 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e930f3c3f4549879221babe49e1cffc', 'request_id': 'req-68df5e19-a347-43f9-9901-0c93e1abb10c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:11.676413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 957/1892] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:11 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:11.680920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f39391-3958-4e7c-8631-08fb95813160 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:11.681069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e930f3c3f4549879221babe49e1cffc', 'request_id': 'req-68df5e19-a347-43f9-9901-0c93e1abb10c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:11.694248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e930f3c3f4549879221babe49e1cffc', 'request_id': 'req-68df5e19-a347-43f9-9901-0c93e1abb10c'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:11.717035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] The token cache is being invalidate because project 8e930f3c3f4549879221babe49e1cffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:11.717327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:11.724012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:11.730021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 936/1893] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:11 2022] DELETE /identity/v3/projects/8e930f3c3f4549879221babe49e1cffc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:11.731405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:11.734421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:11.780681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:11.855009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f39391-3958-4e7c-8631-08fb95813160 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:11.876657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4a92e-8af8-4531-8df8-89b3f47e3067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:11.877469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4a92e-8af8-4531-8df8-89b3f47e3067 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:11.877791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4a92e-8af8-4531-8df8-89b3f47e3067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:11.878072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4a92e-8af8-4531-8df8-89b3f47e3067 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:11.879219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4a92e-8af8-4531-8df8-89b3f47e3067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:11.881555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b4a92e-8af8-4531-8df8-89b3f47e3067 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:11.895952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df5e19-a347-43f9-9901-0c93e1abb10c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:11.907189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:11.911422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:11.913403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:11.913982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] PATH_INFO: `/v3/users/c5986b12bf854bf2a83c260efbad8b96` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:11.917776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5986b12bf854bf2a83c260efbad8b96)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:11.924291 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:11.928303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 958/1894] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:11 2022] POST /identity/v3/users => generated 288 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:11.932258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4a92e-8af8-4531-8df8-89b3f47e3067 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:11.934574 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b5b4a92e-8af8-4531-8df8-89b3f47e3067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:11.962383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4a92e-8af8-4531-8df8-89b3f47e3067 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:11.973067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36c626-191a-46df-8461-bef28bd11c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:11.973713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36c626-191a-46df-8461-bef28bd11c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:11.974027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36c626-191a-46df-8461-bef28bd11c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:11.974300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36c626-191a-46df-8461-bef28bd11c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:11.975073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36c626-191a-46df-8461-bef28bd11c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:11.976603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36c626-191a-46df-8461-bef28bd11c2a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:12.002911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 959/1895] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:12.014941 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36c626-191a-46df-8461-bef28bd11c2a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:12.032646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36c626-191a-46df-8461-bef28bd11c2a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:12.038506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830d608-4106-4e5c-bced-77e4dfa51ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:12.039149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830d608-4106-4e5c-bced-77e4dfa51ae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:12.039425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830d608-4106-4e5c-bced-77e4dfa51ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:12.039697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830d608-4106-4e5c-bced-77e4dfa51ae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:12.045547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830d608-4106-4e5c-bced-77e4dfa51ae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:12.048035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830d608-4106-4e5c-bced-77e4dfa51ae1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:12.065460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5986b12bf854bf2a83c260efbad8b96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:12.066829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5986b12bf854bf2a83c260efbad8b96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:12.078664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 960/1896] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:12.083628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830d608-4106-4e5c-bced-77e4dfa51ae1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:12.098148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830d608-4106-4e5c-bced-77e4dfa51ae1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:12.102119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5986b12bf854bf2a83c260efbad8b96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:12.105446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4255d3-9e16-4712-8baf-3a9dac57dd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:12.106513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4255d3-9e16-4712-8baf-3a9dac57dd18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:12.107655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4255d3-9e16-4712-8baf-3a9dac57dd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:12.107655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4255d3-9e16-4712-8baf-3a9dac57dd18 admin admin] PATH_INFO: `/v3/projects/23e38dd0066145199fd481e34efec5b9/users/1adf8f7ce25f488ba98012cecd581cbf/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:12.113460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5986b12bf854bf2a83c260efbad8b96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:12.149627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5986b12bf854bf2a83c260efbad8b96'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:12.155575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4255d3-9e16-4712-8baf-3a9dac57dd18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e38dd0066145199fd481e34efec5b9, user_id=1adf8f7ce25f488ba98012cecd581cbf, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:12.162338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4255d3-9e16-4712-8baf-3a9dac57dd18 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:12.167844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 937/1897] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:11 2022] DELETE /identity/v3/users/c5986b12bf854bf2a83c260efbad8b96 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:12.174706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:12.178686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:12.190724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 961/1898] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:12 2022] PUT /identity/v3/projects/23e38dd0066145199fd481e34efec5b9/users/1adf8f7ce25f488ba98012cecd581cbf/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:12.201326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec343-b68b-4750-a736-48fb3059d209 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:12.201792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec343-b68b-4750-a736-48fb3059d209 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:12.202311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ec343-b68b-4750-a736-48fb3059d209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:12.205263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:12.266571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e85ec343-b68b-4750-a736-48fb3059d209 None None] MFA Rules not processed for user `1adf8f7ce25f488ba98012cecd581cbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:12.271625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e85ec343-b68b-4750-a736-48fb3059d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:12.315529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a03a3f-5a3f-4308-b9ca-676a96247394 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:12.420905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 962/1899] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:12.448677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ec343-b68b-4750-a736-48fb3059d209 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:12.450889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e85ec343-b68b-4750-a736-48fb3059d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:12.512492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:12.514747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:12.515710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:12.516518 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] PATH_INFO: `/v3/users/a2d77afdefec41bebe2eb3012d927aae` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:12.525660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d77afdefec41bebe2eb3012d927aae)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:12.530426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:12.605712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ec343-b68b-4750-a736-48fb3059d209 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:12.625778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1365e7de-5788-477f-af56-cd1829240399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:12.626737 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365e7de-5788-477f-af56-cd1829240399 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:12.627019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365e7de-5788-477f-af56-cd1829240399 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:12.627309 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1365e7de-5788-477f-af56-cd1829240399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:12.673313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d77afdefec41bebe2eb3012d927aae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:12.674418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365e7de-5788-477f-af56-cd1829240399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:12.675959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1365e7de-5788-477f-af56-cd1829240399 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:12.681903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d77afdefec41bebe2eb3012d927aae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:12.712638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 963/1900] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:12.739078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d77afdefec41bebe2eb3012d927aae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:12.760166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d77afdefec41bebe2eb3012d927aae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:12.781827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d77afdefec41bebe2eb3012d927aae'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:12.798779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 938/1901] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:12 2022] DELETE /identity/v3/users/a2d77afdefec41bebe2eb3012d927aae => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:12.803650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd25945-67c4-446d-808d-a6f45e93d69a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:12.803944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd25945-67c4-446d-808d-a6f45e93d69a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:12.804223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd25945-67c4-446d-808d-a6f45e93d69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:12.828551 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6cd25945-67c4-446d-808d-a6f45e93d69a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:12.830259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6cd25945-67c4-446d-808d-a6f45e93d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:12.866155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:12.867567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:12.875917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:12.955237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:13.003339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 964/1902] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:13.006000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781e315c-32da-400a-a8ba-14d332005692 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:13.011482 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:13.012159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.012438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.012718 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] PATH_INFO: `/v3/projects/3e4d00a67f6b4f9c8ff1fa3bd98b0b9f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.014019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4d00a67f6b4f9c8ff1fa3bd98b0b9f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:13.016085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd25945-67c4-446d-808d-a6f45e93d69a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:13.017481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6cd25945-67c4-446d-808d-a6f45e93d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:13.029395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:13.130269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd25945-67c4-446d-808d-a6f45e93d69a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:13.133037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4d00a67f6b4f9c8ff1fa3bd98b0b9f', 'request_id': 'req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:13.145997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4d00a67f6b4f9c8ff1fa3bd98b0b9f', 'request_id': 'req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:13.161583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4d00a67f6b4f9c8ff1fa3bd98b0b9f', 'request_id': 'req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:13.206900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] The token cache is being invalidate because project 3e4d00a67f6b4f9c8ff1fa3bd98b0b9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:13.206900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:13.206900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0b3270ba-c7cf-4c30-8ef6-ccd97d4b5500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:13.211069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 939/1903] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:12 2022] DELETE /identity/v3/projects/3e4d00a67f6b4f9c8ff1fa3bd98b0b9f => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:13.214396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ead64-b6e0-44bd-b26f-fa1bf47dce2d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.219823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ead64-b6e0-44bd-b26f-fa1bf47dce2d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.220246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ead64-b6e0-44bd-b26f-fa1bf47dce2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.253759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7d894-23dc-4988-ab15-8c71c2136cb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:13.254718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7d894-23dc-4988-ab15-8c71c2136cb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.255149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7d894-23dc-4988-ab15-8c71c2136cb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.255594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7d894-23dc-4988-ab15-8c71c2136cb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.260621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-65e7d894-23dc-4988-ab15-8c71c2136cb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:13.277986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-458ead64-b6e0-44bd-b26f-fa1bf47dce2d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:13.279876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-458ead64-b6e0-44bd-b26f-fa1bf47dce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:13.398481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7d894-23dc-4988-ab15-8c71c2136cb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:13.400680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7d894-23dc-4988-ab15-8c71c2136cb3 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:13.410839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 940/1904] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:13.410839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ead64-b6e0-44bd-b26f-fa1bf47dce2d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:13.410839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-458ead64-b6e0-44bd-b26f-fa1bf47dce2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:13.420907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 965/1905] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:13.422225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3a097-3e42-47f0-9b11-9f1ac82bb335 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.422510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3a097-3e42-47f0-9b11-9f1ac82bb335 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.422782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3a097-3e42-47f0-9b11-9f1ac82bb335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.441689 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ead64-b6e0-44bd-b26f-fa1bf47dce2d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:13.447846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:13.448733 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.449333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.450324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] PATH_INFO: `/v3/users/2e8cf602969c4d83bbbf71333c39ae87` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.452699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e8cf602969c4d83bbbf71333c39ae87)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:13.455178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:13.461136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-73b3a097-3e42-47f0-9b11-9f1ac82bb335 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:13.466571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-73b3a097-3e42-47f0-9b11-9f1ac82bb335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:13.492536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 966/1906] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:13.494158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3a097-3e42-47f0-9b11-9f1ac82bb335 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:13.511299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3a097-3e42-47f0-9b11-9f1ac82bb335 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:13.520052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbab16a-797a-4712-8bc6-a60844b51bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:13.521319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbab16a-797a-4712-8bc6-a60844b51bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.521603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbab16a-797a-4712-8bc6-a60844b51bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.521917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbab16a-797a-4712-8bc6-a60844b51bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.523833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbab16a-797a-4712-8bc6-a60844b51bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:13.524323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbab16a-797a-4712-8bc6-a60844b51bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:13.527147 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbab16a-797a-4712-8bc6-a60844b51bc3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:13.543784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:13.552113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e8cf602969c4d83bbbf71333c39ae87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:13.553108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e8cf602969c4d83bbbf71333c39ae87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:13.557806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 967/1907] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:13.561647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbab16a-797a-4712-8bc6-a60844b51bc3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:13.573406 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbab16a-797a-4712-8bc6-a60844b51bc3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:13.574436 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:13.578051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e8cf602969c4d83bbbf71333c39ae87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:13.584960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf36aa-aede-4b80-919f-4d28a9bc60a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:13.585990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf36aa-aede-4b80-919f-4d28a9bc60a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.586634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf36aa-aede-4b80-919f-4d28a9bc60a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.587088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf36aa-aede-4b80-919f-4d28a9bc60a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.589120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf36aa-aede-4b80-919f-4d28a9bc60a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:13.591150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bf36aa-aede-4b80-919f-4d28a9bc60a5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:13.591434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e8cf602969c4d83bbbf71333c39ae87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:13.603636 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3a9d1bf-097e-4e9d-b4e5-8a8574a4b9bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e8cf602969c4d83bbbf71333c39ae87'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:13.613569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 941/1908] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:13 2022] DELETE /identity/v3/users/2e8cf602969c4d83bbbf71333c39ae87 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:13.619016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97714028-178b-4d15-95f1-453653d1e85e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.620407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97714028-178b-4d15-95f1-453653d1e85e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.620866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97714028-178b-4d15-95f1-453653d1e85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.636097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 968/1909] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:13 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:13.640321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf36aa-aede-4b80-919f-4d28a9bc60a5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:13.643233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-97bf36aa-aede-4b80-919f-4d28a9bc60a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:13.678739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-97714028-178b-4d15-95f1-453653d1e85e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:13.681998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97714028-178b-4d15-95f1-453653d1e85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:13.772490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bf36aa-aede-4b80-919f-4d28a9bc60a5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:13.780548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f85ae-3706-4751-b5c3-c79ebcbfa6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:13.781270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f85ae-3706-4751-b5c3-c79ebcbfa6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.782130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f85ae-3706-4751-b5c3-c79ebcbfa6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.782130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f85ae-3706-4751-b5c3-c79ebcbfa6e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.782865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f85ae-3706-4751-b5c3-c79ebcbfa6e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:13.785296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 942/1910] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:13.788410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f85ae-3706-4751-b5c3-c79ebcbfa6e2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:13.816862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97714028-178b-4d15-95f1-453653d1e85e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:13.818541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-97714028-178b-4d15-95f1-453653d1e85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:13.840423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 969/1911] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:13 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:13.846224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f85ae-3706-4751-b5c3-c79ebcbfa6e2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:13.867857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f85ae-3706-4751-b5c3-c79ebcbfa6e2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:13.874340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d01de1-9ee9-4390-81c3-9cd776672e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:13.875642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d01de1-9ee9-4390-81c3-9cd776672e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.876947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d01de1-9ee9-4390-81c3-9cd776672e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.877461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d01de1-9ee9-4390-81c3-9cd776672e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.878620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d01de1-9ee9-4390-81c3-9cd776672e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:13.883271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d01de1-9ee9-4390-81c3-9cd776672e34 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:13.909975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 970/1912] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:13.914868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d01de1-9ee9-4390-81c3-9cd776672e34 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:13.918699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97714028-178b-4d15-95f1-453653d1e85e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:13.919974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-46d01de1-9ee9-4390-81c3-9cd776672e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:13.928460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79b5ae-11b2-4402-ab4b-c36f3ad96c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:13.930222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79b5ae-11b2-4402-ab4b-c36f3ad96c16 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.939070 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d01de1-9ee9-4390-81c3-9cd776672e34 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:13.942281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79b5ae-11b2-4402-ab4b-c36f3ad96c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.943150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d79b5ae-11b2-4402-ab4b-c36f3ad96c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.944180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:13.944966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.945347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:13.945719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] PATH_INFO: `/v3/projects/16c8c18654c249f08075779ceb4c4fc0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:13.947170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c8c18654c249f08075779ceb4c4fc0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:13.949427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:13.959625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79b5ae-11b2-4402-ab4b-c36f3ad96c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:13.961528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d79b5ae-11b2-4402-ab4b-c36f3ad96c16 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:13.974824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 943/1913] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:13.975939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79b5ae-11b2-4402-ab4b-c36f3ad96c16 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:13.990925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d79b5ae-11b2-4402-ab4b-c36f3ad96c16 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:13.997080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f86e7-425f-4402-9643-53f05267d834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:13.999061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f86e7-425f-4402-9643-53f05267d834 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:13.999589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f86e7-425f-4402-9643-53f05267d834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:14.000003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f86e7-425f-4402-9643-53f05267d834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:14.001040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f86e7-425f-4402-9643-53f05267d834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:14.003317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f86e7-425f-4402-9643-53f05267d834 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:14.013974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c8c18654c249f08075779ceb4c4fc0', 'request_id': 'req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.017115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 944/1914] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:14.023018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c8c18654c249f08075779ceb4c4fc0', 'request_id': 'req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.023318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f86e7-425f-4402-9643-53f05267d834 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:14.030728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c8c18654c249f08075779ceb4c4fc0', 'request_id': 'req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.036637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f86e7-425f-4402-9643-53f05267d834 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:14.051171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] The token cache is being invalidate because project 16c8c18654c249f08075779ceb4c4fc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:14.051833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.052819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.054807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 971/1915] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:13 2022] DELETE /identity/v3/projects/16c8c18654c249f08075779ceb4c4fc0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:14.059290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:14.061287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:14.116188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:14.144577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20a7ac-2dd3-4da3-8947-78a7cde4c21f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:14.150704 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:14.150923 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:14.151196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:14.151466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] PATH_INFO: `/v3/users/d53ad1397e084613b500892ae9f34e7d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:14.152701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=d53ad1397e084613b500892ae9f34e7d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:14.154934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:14.185686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e959b7-51d0-4712-b0eb-1e8034bcf636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:14.186418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e959b7-51d0-4712-b0eb-1e8034bcf636 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:14.186703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e959b7-51d0-4712-b0eb-1e8034bcf636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:14.187023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e959b7-51d0-4712-b0eb-1e8034bcf636 admin admin] PATH_INFO: `/v3/projects/e7a428c7a603497b96c527f72aa33722/users/8f1fcc1f5aea458db9c96e17478f390d/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:14.216694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e959b7-51d0-4712-b0eb-1e8034bcf636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a428c7a603497b96c527f72aa33722, user_id=8f1fcc1f5aea458db9c96e17478f390d, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:14.220055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e959b7-51d0-4712-b0eb-1e8034bcf636 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:14.223939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:14.264049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd53ad1397e084613b500892ae9f34e7d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.264443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 945/1916] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:14 2022] PUT /identity/v3/projects/e7a428c7a603497b96c527f72aa33722/users/8f1fcc1f5aea458db9c96e17478f390d/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:14.264771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd53ad1397e084613b500892ae9f34e7d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.275438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6be8ab-96c3-41f7-b49e-65cfd9c3d325 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:14.275968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6be8ab-96c3-41f7-b49e-65cfd9c3d325 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:14.276249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6be8ab-96c3-41f7-b49e-65cfd9c3d325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:14.320485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd53ad1397e084613b500892ae9f34e7d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.330748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd53ad1397e084613b500892ae9f34e7d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.346058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd53ad1397e084613b500892ae9f34e7d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.374357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 972/1917] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:14 2022] DELETE /identity/v3/users/d53ad1397e084613b500892ae9f34e7d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:14.404167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-be6be8ab-96c3-41f7-b49e-65cfd9c3d325 None None] MFA Rules not processed for user `8f1fcc1f5aea458db9c96e17478f390d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:14.406050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be6be8ab-96c3-41f7-b49e-65cfd9c3d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:14.585817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:14.587511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:14.630084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 946/1918] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:14.634437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6be8ab-96c3-41f7-b49e-65cfd9c3d325 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:14.637590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-be6be8ab-96c3-41f7-b49e-65cfd9c3d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:14.808000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3341ab-7580-48c7-9783-407a011b5aee admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:14.823323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:14.824221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:14.824856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:14.825637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] PATH_INFO: `/v3/projects/df3d1d84fc6d430f8506c7578bbe61fd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:14.828026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=df3d1d84fc6d430f8506c7578bbe61fd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:14.831915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:14.884894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6be8ab-96c3-41f7-b49e-65cfd9c3d325 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:14.894236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e458e-a325-4e73-8a80-6e40ec56684c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:14.894974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e458e-a325-4e73-8a80-6e40ec56684c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:14.895332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e458e-a325-4e73-8a80-6e40ec56684c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:14.895722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e458e-a325-4e73-8a80-6e40ec56684c admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:14.896429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e458e-a325-4e73-8a80-6e40ec56684c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:14.902731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e458e-a325-4e73-8a80-6e40ec56684c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:14.938184 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df3d1d84fc6d430f8506c7578bbe61fd', 'request_id': 'req-cff096b7-5977-43de-acb3-2d8bbf91c8ed'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.959368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df3d1d84fc6d430f8506c7578bbe61fd', 'request_id': 'req-cff096b7-5977-43de-acb3-2d8bbf91c8ed'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:14.966226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 947/1919] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:14 2022] POST /identity/v3/projects => generated 379 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:14.971065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e458e-a325-4e73-8a80-6e40ec56684c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:14.973576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:14.980426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df3d1d84fc6d430f8506c7578bbe61fd', 'request_id': 'req-cff096b7-5977-43de-acb3-2d8bbf91c8ed'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:15.024617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] The token cache is being invalidate because project df3d1d84fc6d430f8506c7578bbe61fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:15.025063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:15.032114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:15.042471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 973/1920] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:14 2022] DELETE /identity/v3/projects/df3d1d84fc6d430f8506c7578bbe61fd => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:15.051166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:15.053947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:15.109062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e458e-a325-4e73-8a80-6e40ec56684c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:15.117534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeec561-7d37-4a09-b3c0-af22d2de562d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:15.118914 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeec561-7d37-4a09-b3c0-af22d2de562d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:15.119131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeec561-7d37-4a09-b3c0-af22d2de562d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:15.119542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeec561-7d37-4a09-b3c0-af22d2de562d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:15.120259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeec561-7d37-4a09-b3c0-af22d2de562d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:15.138944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeec561-7d37-4a09-b3c0-af22d2de562d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:15.176024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:15.194733 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 948/1921] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:14 2022] POST /identity/v3/users => generated 290 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:15.199316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeec561-7d37-4a09-b3c0-af22d2de562d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:15.202853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bbeec561-7d37-4a09-b3c0-af22d2de562d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:15.248688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeec561-7d37-4a09-b3c0-af22d2de562d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:15.262821 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb35b4f-b587-4b0e-9790-7610cd40f1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:15.264757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb35b4f-b587-4b0e-9790-7610cd40f1df admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:15.265060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb35b4f-b587-4b0e-9790-7610cd40f1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:15.265346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb35b4f-b587-4b0e-9790-7610cd40f1df admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:15.266171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb35b4f-b587-4b0e-9790-7610cd40f1df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:15.278650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb35b4f-b587-4b0e-9790-7610cd40f1df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:15.299298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 949/1922] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:15.305912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb35b4f-b587-4b0e-9790-7610cd40f1df admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:15.320101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb35b4f-b587-4b0e-9790-7610cd40f1df admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:15.326104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e96498-6469-477c-899f-23e0a21cdf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:15.326773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e96498-6469-477c-899f-23e0a21cdf7b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:15.327061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e96498-6469-477c-899f-23e0a21cdf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:15.327478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e96498-6469-477c-899f-23e0a21cdf7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:15.328203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e96498-6469-477c-899f-23e0a21cdf7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:15.330185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e96498-6469-477c-899f-23e0a21cdf7b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:15.343164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff096b7-5977-43de-acb3-2d8bbf91c8ed admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:15.348084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 950/1923] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:15.354811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e96498-6469-477c-899f-23e0a21cdf7b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:15.360087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:15.360770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:15.361052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:15.361327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] PATH_INFO: `/v3/users/e187533e8a39464e814b542e0eaabb81` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:15.362806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e187533e8a39464e814b542e0eaabb81)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:15.364571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:15.372487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e96498-6469-477c-899f-23e0a21cdf7b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:15.391162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1752676-33f8-46ed-a227-6540943517ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:15.391872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1752676-33f8-46ed-a227-6540943517ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:15.392188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1752676-33f8-46ed-a227-6540943517ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:15.392471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1752676-33f8-46ed-a227-6540943517ba admin admin] PATH_INFO: `/v3/projects/8c6123799f7443a38e861a0392f2ae93/users/35252113721d4748b6eb93a7ff1c4c23/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:15.444492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1752676-33f8-46ed-a227-6540943517ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c6123799f7443a38e861a0392f2ae93, user_id=35252113721d4748b6eb93a7ff1c4c23, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:15.447924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1752676-33f8-46ed-a227-6540943517ba admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:15.472045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 951/1924] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:15 2022] PUT /identity/v3/projects/8c6123799f7443a38e861a0392f2ae93/users/35252113721d4748b6eb93a7ff1c4c23/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:15.485982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714301f3-02ed-4083-80ac-82845658ff71 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:15.486950 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714301f3-02ed-4083-80ac-82845658ff71 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:15.487247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714301f3-02ed-4083-80ac-82845658ff71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:15.515132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e187533e8a39464e814b542e0eaabb81'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:15.517394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e187533e8a39464e814b542e0eaabb81'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:15.551835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e187533e8a39464e814b542e0eaabb81'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:15.573727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e187533e8a39464e814b542e0eaabb81'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:15.582794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-714301f3-02ed-4083-80ac-82845658ff71 None None] MFA Rules not processed for user `35252113721d4748b6eb93a7ff1c4c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:15.584047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-714301f3-02ed-4083-80ac-82845658ff71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:15.594826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e187533e8a39464e814b542e0eaabb81'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:15.605415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 974/1925] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:15 2022] DELETE /identity/v3/users/e187533e8a39464e814b542e0eaabb81 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:15.759207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:15.760670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:15.786204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 952/1926] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:15.790778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714301f3-02ed-4083-80ac-82845658ff71 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:15.792298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-714301f3-02ed-4083-80ac-82845658ff71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:15.915161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cca84ca-6243-4431-b6ea-fa759352cc18 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:15.922341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:15.923836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:15.924107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:15.924721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] PATH_INFO: `/v3/projects/aeeac2d781e94d5d9b9bc3a7820f5f41` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:15.926299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeeac2d781e94d5d9b9bc3a7820f5f41)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:15.928653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:15.940482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:15.954757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714301f3-02ed-4083-80ac-82845658ff71 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:15.979259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171ef01-3736-4583-83e0-a2ed7b7e13ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:15.979968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171ef01-3736-4583-83e0-a2ed7b7e13ac admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:15.980469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171ef01-3736-4583-83e0-a2ed7b7e13ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:15.980882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8171ef01-3736-4583-83e0-a2ed7b7e13ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:15.981549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171ef01-3736-4583-83e0-a2ed7b7e13ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:15.983618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8171ef01-3736-4583-83e0-a2ed7b7e13ac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.002119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeeac2d781e94d5d9b9bc3a7820f5f41', 'request_id': 'req-c548016a-b287-47d5-8dab-8dc842d4b2a4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:16.009791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeeac2d781e94d5d9b9bc3a7820f5f41', 'request_id': 'req-c548016a-b287-47d5-8dab-8dc842d4b2a4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:16.019372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeeac2d781e94d5d9b9bc3a7820f5f41', 'request_id': 'req-c548016a-b287-47d5-8dab-8dc842d4b2a4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:16.045817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 953/1927] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:15 2022] POST /identity/v3/projects => generated 377 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:16.052416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171ef01-3736-4583-83e0-a2ed7b7e13ac admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.052694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] The token cache is being invalidate because project aeeac2d781e94d5d9b9bc3a7820f5f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:16.052997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:16.054047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c548016a-b287-47d5-8dab-8dc842d4b2a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:16.061329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 975/1928] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:15 2022] DELETE /identity/v3/projects/aeeac2d781e94d5d9b9bc3a7820f5f41 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:16.063270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf816c37-0f83-467b-812f-e3297e3921e9 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.063572 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf816c37-0f83-467b-812f-e3297e3921e9 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.064059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf816c37-0f83-467b-812f-e3297e3921e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.104696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-bf816c37-0f83-467b-812f-e3297e3921e9 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:16.106582 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bf816c37-0f83-467b-812f-e3297e3921e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:16.114812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:16.163352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8171ef01-3736-4583-83e0-a2ed7b7e13ac admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.170156 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38510f60-b7e5-4482-a121-39c546e34c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.170848 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38510f60-b7e5-4482-a121-39c546e34c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.171133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38510f60-b7e5-4482-a121-39c546e34c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.171532 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38510f60-b7e5-4482-a121-39c546e34c34 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.172452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38510f60-b7e5-4482-a121-39c546e34c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:16.174447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38510f60-b7e5-4482-a121-39c546e34c34 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.210651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 954/1929] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:16 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:16.214420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38510f60-b7e5-4482-a121-39c546e34c34 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.218208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-38510f60-b7e5-4482-a121-39c546e34c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:16.221223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 976/1930] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:16.230692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f3266-ffcd-4679-9d39-9fd497f5badb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.230692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f3266-ffcd-4679-9d39-9fd497f5badb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.231001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f3266-ffcd-4679-9d39-9fd497f5badb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.236373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38510f60-b7e5-4482-a121-39c546e34c34 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.243028 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c5acc-393d-4d75-9695-e139e7c2ee13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.244178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c5acc-393d-4d75-9695-e139e7c2ee13 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.244663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c5acc-393d-4d75-9695-e139e7c2ee13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.245129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c5acc-393d-4d75-9695-e139e7c2ee13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.246351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c5acc-393d-4d75-9695-e139e7c2ee13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:16.248478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93c5acc-393d-4d75-9695-e139e7c2ee13 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.264812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 955/1931] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:16.266403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c5acc-393d-4d75-9695-e139e7c2ee13 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.277062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-411f3266-ffcd-4679-9d39-9fd497f5badb None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:16.277777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c5acc-393d-4d75-9695-e139e7c2ee13 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.279243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-411f3266-ffcd-4679-9d39-9fd497f5badb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:16.286049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a0fcc-f849-4ee5-9113-0b212c5a7870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.287184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a0fcc-f849-4ee5-9113-0b212c5a7870 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.287891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a0fcc-f849-4ee5-9113-0b212c5a7870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.288454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a0fcc-f849-4ee5-9113-0b212c5a7870 admin admin] PATH_INFO: `/v3/projects/2e846b7fa81a4524b311e58e215b9939/users/62637f2d901f4bb48d1811c5f05d13c9/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.289976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 977/1932] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:16.296090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f3266-ffcd-4679-9d39-9fd497f5badb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.307270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411f3266-ffcd-4679-9d39-9fd497f5badb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.313048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1b8d7-2358-4102-9c06-8fcb5a2b1f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.314054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1b8d7-2358-4102-9c06-8fcb5a2b1f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.314785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1b8d7-2358-4102-9c06-8fcb5a2b1f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.315233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1b8d7-2358-4102-9c06-8fcb5a2b1f58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.316087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1b8d7-2358-4102-9c06-8fcb5a2b1f58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:16.316533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1b8d7-2358-4102-9c06-8fcb5a2b1f58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:16.319012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a0fcc-f849-4ee5-9113-0b212c5a7870 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e846b7fa81a4524b311e58e215b9939, user_id=62637f2d901f4bb48d1811c5f05d13c9, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:16.319243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a1b8d7-2358-4102-9c06-8fcb5a2b1f58 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.323111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a0fcc-f849-4ee5-9113-0b212c5a7870 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.337788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 978/1933] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:16.342821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1b8d7-2358-4102-9c06-8fcb5a2b1f58 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.350780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 956/1934] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:16 2022] PUT /identity/v3/projects/2e846b7fa81a4524b311e58e215b9939/users/62637f2d901f4bb48d1811c5f05d13c9/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:16.353452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a0fcc-f849-4ee5-9113-0b212c5a7870 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.356227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1b8d7-2358-4102-9c06-8fcb5a2b1f58 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.459337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a0fcc-f849-4ee5-9113-0b212c5a7870 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.459869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24093f-632e-4f11-8723-e04cc6bb81ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.460530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24093f-632e-4f11-8723-e04cc6bb81ec admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.460867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24093f-632e-4f11-8723-e04cc6bb81ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.461084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce24093f-632e-4f11-8723-e04cc6bb81ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.462162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24093f-632e-4f11-8723-e04cc6bb81ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:16.464126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce24093f-632e-4f11-8723-e04cc6bb81ec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.466332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38432ab-adde-411c-8189-0c0c3dbd25be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.467116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38432ab-adde-411c-8189-0c0c3dbd25be admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.467488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38432ab-adde-411c-8189-0c0c3dbd25be admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.467818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38432ab-adde-411c-8189-0c0c3dbd25be admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.468730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38432ab-adde-411c-8189-0c0c3dbd25be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:16.471050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38432ab-adde-411c-8189-0c0c3dbd25be admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.505179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 957/1935] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:16.510070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38432ab-adde-411c-8189-0c0c3dbd25be admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.528555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38432ab-adde-411c-8189-0c0c3dbd25be admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.534408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5fe663-4ec0-45f7-abc6-c6578f305d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.535617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe663-4ec0-45f7-abc6-c6578f305d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.536743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe663-4ec0-45f7-abc6-c6578f305d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.536743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5fe663-4ec0-45f7-abc6-c6578f305d93 admin admin] PATH_INFO: `/v3/projects/2e846b7fa81a4524b311e58e215b9939/users/62637f2d901f4bb48d1811c5f05d13c9/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.537364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 979/1936] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:16 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:16.539924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fe663-4ec0-45f7-abc6-c6578f305d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e846b7fa81a4524b311e58e215b9939, user_id=62637f2d901f4bb48d1811c5f05d13c9, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:16.543196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24093f-632e-4f11-8723-e04cc6bb81ec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.549490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5fe663-4ec0-45f7-abc6-c6578f305d93 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.563211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:16.580327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 958/1937] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:16 2022] PUT /identity/v3/projects/2e846b7fa81a4524b311e58e215b9939/users/62637f2d901f4bb48d1811c5f05d13c9/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:16.592914 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adde50e-96ea-4779-901b-aed6468a69a5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.593226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adde50e-96ea-4779-901b-aed6468a69a5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.593503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adde50e-96ea-4779-901b-aed6468a69a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.648926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2adde50e-96ea-4779-901b-aed6468a69a5 None None] MFA Rules not processed for user `62637f2d901f4bb48d1811c5f05d13c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:16.649901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2adde50e-96ea-4779-901b-aed6468a69a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:16.662590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce24093f-632e-4f11-8723-e04cc6bb81ec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.715852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:16.725806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71213f57-39b2-4135-b6ab-b797d9433d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.726695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71213f57-39b2-4135-b6ab-b797d9433d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.727391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71213f57-39b2-4135-b6ab-b797d9433d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.727391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71213f57-39b2-4135-b6ab-b797d9433d1f admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.728014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71213f57-39b2-4135-b6ab-b797d9433d1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:16.730233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71213f57-39b2-4135-b6ab-b797d9433d1f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.738485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 959/1938] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:16.739562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adde50e-96ea-4779-901b-aed6468a69a5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.741246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2adde50e-96ea-4779-901b-aed6468a69a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:16.758011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adde50e-96ea-4779-901b-aed6468a69a5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.761829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 980/1939] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:16 2022] POST /identity/v3/users => generated 292 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:16.764332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.764332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.764332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.764663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] PATH_INFO: `/v3/projects/b6165a1dd1164cf0856dfc34946311f6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.765589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71213f57-39b2-4135-b6ab-b797d9433d1f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.766161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6165a1dd1164cf0856dfc34946311f6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:16.767918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-71213f57-39b2-4135-b6ab-b797d9433d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:16.768099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.839425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6165a1dd1164cf0856dfc34946311f6', 'request_id': 'req-7d97de50-e4f7-433b-ad74-1d8cd8196f57'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:16.846161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6165a1dd1164cf0856dfc34946311f6', 'request_id': 'req-7d97de50-e4f7-433b-ad74-1d8cd8196f57'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:16.855743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6165a1dd1164cf0856dfc34946311f6', 'request_id': 'req-7d97de50-e4f7-433b-ad74-1d8cd8196f57'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:16.861787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71213f57-39b2-4135-b6ab-b797d9433d1f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.871765 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] The token cache is being invalidate because project b6165a1dd1164cf0856dfc34946311f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:16.872056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:16.874238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:16.876856 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 960/1940] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:16 2022] DELETE /identity/v3/projects/b6165a1dd1164cf0856dfc34946311f6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:16.877240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:16.882777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:16.926927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a694706-a550-4db9-8a55-fcb767b7961a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.928319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a694706-a550-4db9-8a55-fcb767b7961a service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.928796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a694706-a550-4db9-8a55-fcb767b7961a service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.929285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a694706-a550-4db9-8a55-fcb767b7961a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.933538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6a694706-a550-4db9-8a55-fcb767b7961a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:16.943584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:16.956940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97de50-e4f7-433b-ad74-1d8cd8196f57 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:16.968378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed9075-b3c3-4464-a790-f846671d0235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:16.968793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed9075-b3c3-4464-a790-f846671d0235 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.969440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed9075-b3c3-4464-a790-f846671d0235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.969716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed9075-b3c3-4464-a790-f846671d0235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:16.970464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed9075-b3c3-4464-a790-f846671d0235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:16.972109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed9075-b3c3-4464-a790-f846671d0235 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:16.982640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 961/1941] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:16.983881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e65ad-ce7e-4eec-be8e-72fc6fcc3123 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:16.984167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e65ad-ce7e-4eec-be8e-72fc6fcc3123 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:16.984430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e65ad-ce7e-4eec-be8e-72fc6fcc3123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.002597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a694706-a550-4db9-8a55-fcb767b7961a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.004018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-729e65ad-ce7e-4eec-be8e-72fc6fcc3123 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:17.004175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a694706-a550-4db9-8a55-fcb767b7961a service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.005228 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-729e65ad-ce7e-4eec-be8e-72fc6fcc3123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.014829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 962/1942] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:17.015207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 981/1943] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:16 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:17.017847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e65ad-ce7e-4eec-be8e-72fc6fcc3123 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.024094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8c9d-8bbe-4fae-930a-31cd52f81292 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.024491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8c9d-8bbe-4fae-930a-31cd52f81292 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.024779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c8c9d-8bbe-4fae-930a-31cd52f81292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.027783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e65ad-ce7e-4eec-be8e-72fc6fcc3123 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.032355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96448d-a57d-4fb2-ab6e-81a160b1b833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.032989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96448d-a57d-4fb2-ab6e-81a160b1b833 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.033606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96448d-a57d-4fb2-ab6e-81a160b1b833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.033606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96448d-a57d-4fb2-ab6e-81a160b1b833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.034449 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96448d-a57d-4fb2-ab6e-81a160b1b833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.036071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96448d-a57d-4fb2-ab6e-81a160b1b833 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.045751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c53c8c9d-8bbe-4fae-930a-31cd52f81292 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:17.046896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c53c8c9d-8bbe-4fae-930a-31cd52f81292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.047357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 963/1944] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:17.051602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96448d-a57d-4fb2-ab6e-81a160b1b833 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.061524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 982/1945] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:17.062436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c8c9d-8bbe-4fae-930a-31cd52f81292 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.062558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96448d-a57d-4fb2-ab6e-81a160b1b833 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.063792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c53c8c9d-8bbe-4fae-930a-31cd52f81292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.067313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d2a24-754d-44f0-a71e-cff34bd10aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.067987 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d2a24-754d-44f0-a71e-cff34bd10aea admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.068298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d2a24-754d-44f0-a71e-cff34bd10aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.068572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4d2a24-754d-44f0-a71e-cff34bd10aea admin admin] PATH_INFO: `/v3/projects/e9de6417d89141299453453981c5a4eb/users/0544458d427d402ba3d524d413baed22/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.075736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c8c9d-8bbe-4fae-930a-31cd52f81292 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.079690 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013bd9d-80f6-4b9f-bd14-f475730cfe9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.080325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013bd9d-80f6-4b9f-bd14-f475730cfe9a service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.080585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013bd9d-80f6-4b9f-bd14-f475730cfe9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.080862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013bd9d-80f6-4b9f-bd14-f475730cfe9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.082642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4013bd9d-80f6-4b9f-bd14-f475730cfe9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.084900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d2a24-754d-44f0-a71e-cff34bd10aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9de6417d89141299453453981c5a4eb, user_id=0544458d427d402ba3d524d413baed22, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.087697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4d2a24-754d-44f0-a71e-cff34bd10aea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.102978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 964/1946] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:17 2022] PUT /identity/v3/projects/e9de6417d89141299453453981c5a4eb/users/0544458d427d402ba3d524d413baed22/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:17.103555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d2a24-754d-44f0-a71e-cff34bd10aea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.135164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:17.156234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013bd9d-80f6-4b9f-bd14-f475730cfe9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.157762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013bd9d-80f6-4b9f-bd14-f475730cfe9a service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.166914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 983/1947] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:17.168152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79a1dd-03a9-4458-8c85-bcc8e542b36e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.168436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79a1dd-03a9-4458-8c85-bcc8e542b36e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.168714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79a1dd-03a9-4458-8c85-bcc8e542b36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.169644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4d2a24-754d-44f0-a71e-cff34bd10aea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.174648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be971-4c89-4106-9611-30de6335a4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.175414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be971-4c89-4106-9611-30de6335a4d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.175592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be971-4c89-4106-9611-30de6335a4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.175839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02be971-4c89-4106-9611-30de6335a4d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.176550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02be971-4c89-4106-9611-30de6335a4d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:17.176860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02be971-4c89-4106-9611-30de6335a4d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.178550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02be971-4c89-4106-9611-30de6335a4d2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.195645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 965/1948] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:17.195645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be971-4c89-4106-9611-30de6335a4d2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.197936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b02be971-4c89-4106-9611-30de6335a4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.211934 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02be971-4c89-4106-9611-30de6335a4d2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.212348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ad79a1dd-03a9-4458-8c85-bcc8e542b36e None None] MFA Rules not processed for user `0544458d427d402ba3d524d413baed22`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:17.214203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ad79a1dd-03a9-4458-8c85-bcc8e542b36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.217540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.218177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.219525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.219525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] PATH_INFO: `/v3/users/1adf8f7ce25f488ba98012cecd581cbf` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.220617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1adf8f7ce25f488ba98012cecd581cbf)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.222589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.283664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1adf8f7ce25f488ba98012cecd581cbf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:17.286040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1adf8f7ce25f488ba98012cecd581cbf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:17.307815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1adf8f7ce25f488ba98012cecd581cbf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:17.314155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 984/1949] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:17.315560 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79a1dd-03a9-4458-8c85-bcc8e542b36e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.322964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1adf8f7ce25f488ba98012cecd581cbf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:17.339928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79a1dd-03a9-4458-8c85-bcc8e542b36e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.351037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1adf8f7ce25f488ba98012cecd581cbf'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:17.353121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113a30c-e150-428b-9853-8632540df219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.353820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113a30c-e150-428b-9853-8632540df219 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.354190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113a30c-e150-428b-9853-8632540df219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.354488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113a30c-e150-428b-9853-8632540df219 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.355533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113a30c-e150-428b-9853-8632540df219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.357373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113a30c-e150-428b-9853-8632540df219 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.367919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 966/1950] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:17 2022] DELETE /identity/v3/users/1adf8f7ce25f488ba98012cecd581cbf => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:17.369249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.371327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.423110 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 985/1951] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:17 2022] POST /identity/v3/projects => generated 409 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:17.426192 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffa88d-fc35-4e89-a604-c30e219d1d74 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.426192 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffa88d-fc35-4e89-a604-c30e219d1d74 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.430785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffa88d-fc35-4e89-a604-c30e219d1d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.448896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:17.463820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-20ffa88d-fc35-4e89-a604-c30e219d1d74 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:17.466076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-20ffa88d-fc35-4e89-a604-c30e219d1d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.542107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e6a971-861e-4ff4-b3f1-1cf26651dd31 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.591236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 986/1952] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:17.594387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffa88d-fc35-4e89-a604-c30e219d1d74 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.595142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-20ffa88d-fc35-4e89-a604-c30e219d1d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.610550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffa88d-fc35-4e89-a604-c30e219d1d74 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.616079 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6cba9-62ee-4362-af26-b855221bab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.616883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6cba9-62ee-4362-af26-b855221bab6f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.617301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6cba9-62ee-4362-af26-b855221bab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.617408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1263540-6386-4346-8b25-ce9d6df29cc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.617584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6cba9-62ee-4362-af26-b855221bab6f admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.618076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1263540-6386-4346-8b25-ce9d6df29cc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.618313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6cba9-62ee-4362-af26-b855221bab6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.618452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1263540-6386-4346-8b25-ce9d6df29cc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.618644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1263540-6386-4346-8b25-ce9d6df29cc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.620428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6cba9-62ee-4362-af26-b855221bab6f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.621024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1263540-6386-4346-8b25-ce9d6df29cc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.627701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:17.655608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 987/1953] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:17 2022] POST /identity/v3/users => generated 305 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:17.657346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6cba9-62ee-4362-af26-b855221bab6f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.660247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-22c6cba9-62ee-4362-af26-b855221bab6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.738309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1263540-6386-4346-8b25-ce9d6df29cc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.740897 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1263540-6386-4346-8b25-ce9d6df29cc9 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.743694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6cba9-62ee-4362-af26-b855221bab6f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.757109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 967/1954] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:17.757912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1263540-6386-4346-8b25-ce9d6df29cc9 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.768292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41c490-c16d-4733-bec1-e1abbd4ad26e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.769010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41c490-c16d-4733-bec1-e1abbd4ad26e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.769300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41c490-c16d-4733-bec1-e1abbd4ad26e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.769578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b41c490-c16d-4733-bec1-e1abbd4ad26e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.774727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1263540-6386-4346-8b25-ce9d6df29cc9 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.782672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41c490-c16d-4733-bec1-e1abbd4ad26e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.784623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b41c490-c16d-4733-bec1-e1abbd4ad26e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.786448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ea2fc-c4d8-4a47-a0a5-bb887c9ed1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.787126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ea2fc-c4d8-4a47-a0a5-bb887c9ed1e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.787461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ea2fc-c4d8-4a47-a0a5-bb887c9ed1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.787863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124ea2fc-c4d8-4a47-a0a5-bb887c9ed1e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.793787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ea2fc-c4d8-4a47-a0a5-bb887c9ed1e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.796688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124ea2fc-c4d8-4a47-a0a5-bb887c9ed1e4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.800496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 988/1955] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:17.801980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41c490-c16d-4733-bec1-e1abbd4ad26e service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.804071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0b41c490-c16d-4733-bec1-e1abbd4ad26e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:17.821024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 968/1956] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:17.821154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b41c490-c16d-4733-bec1-e1abbd4ad26e service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.826692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ea2fc-c4d8-4a47-a0a5-bb887c9ed1e4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.827027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.827697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.827977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.828247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] PATH_INFO: `/v3/users/0544458d427d402ba3d524d413baed22` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.829432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0544458d427d402ba3d524d413baed22)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.831401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.837199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124ea2fc-c4d8-4a47-a0a5-bb887c9ed1e4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.845206 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f9043-a92b-4281-b42c-d2f379b885af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.846575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f9043-a92b-4281-b42c-d2f379b885af admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.846861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f9043-a92b-4281-b42c-d2f379b885af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.847139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f9043-a92b-4281-b42c-d2f379b885af admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.848291 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f9043-a92b-4281-b42c-d2f379b885af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.850885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8f9043-a92b-4281-b42c-d2f379b885af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.872773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 969/1957] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:17.874257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f9043-a92b-4281-b42c-d2f379b885af admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:17.887878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f9043-a92b-4281-b42c-d2f379b885af admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:17.895267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a29e1-a470-4f82-8130-eca45fb5180c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:17.896006 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a29e1-a470-4f82-8130-eca45fb5180c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.896297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a29e1-a470-4f82-8130-eca45fb5180c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.896580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a29e1-a470-4f82-8130-eca45fb5180c admin admin] PATH_INFO: `/v3/projects/11e20f95990b4ce2b2c06d3fcdd09b0c/users/a4bf5a09529f478cba31c219471fc84f/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.910403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0544458d427d402ba3d524d413baed22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:17.912422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0544458d427d402ba3d524d413baed22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:17.917025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a29e1-a470-4f82-8130-eca45fb5180c admin admin] RBAC: Authorizing `identity:create_grant(project_id=11e20f95990b4ce2b2c06d3fcdd09b0c, user_id=a4bf5a09529f478cba31c219471fc84f, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:17.924297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a29e1-a470-4f82-8130-eca45fb5180c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:17.929190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0544458d427d402ba3d524d413baed22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:17.938179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0544458d427d402ba3d524d413baed22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:17.952413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-61e318a1-f3ff-4589-bc95-e8dfa347471f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0544458d427d402ba3d524d413baed22'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:17.962636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 989/1958] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:17 2022] DELETE /identity/v3/users/0544458d427d402ba3d524d413baed22 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:17.967491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd54baa-3180-4d1f-9dae-3f67826ba0fe None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.967918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd54baa-3180-4d1f-9dae-3f67826ba0fe None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.968298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd54baa-3180-4d1f-9dae-3f67826ba0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.988613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 970/1959] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:17 2022] PUT /identity/v3/projects/11e20f95990b4ce2b2c06d3fcdd09b0c/users/a4bf5a09529f478cba31c219471fc84f/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:17.990631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a52028-8de0-4ad6-a415-9a02a7d4575f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:17.990927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a52028-8de0-4ad6-a415-9a02a7d4575f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:17.991205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a52028-8de0-4ad6-a415-9a02a7d4575f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:17.998927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:18.014551 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cdd54baa-3180-4d1f-9dae-3f67826ba0fe None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:18.016544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cdd54baa-3180-4d1f-9dae-3f67826ba0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:18.053166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c9a52028-8de0-4ad6-a415-9a02a7d4575f None None] MFA Rules not processed for user `a4bf5a09529f478cba31c219471fc84f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:18.054784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c9a52028-8de0-4ad6-a415-9a02a7d4575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:18.106049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 990/1960] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:18.115596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd54baa-3180-4d1f-9dae-3f67826ba0fe None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:18.117064 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cdd54baa-3180-4d1f-9dae-3f67826ba0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:18.156558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 971/1961] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:17 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:18.160729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a52028-8de0-4ad6-a415-9a02a7d4575f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:18.162526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c9a52028-8de0-4ad6-a415-9a02a7d4575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:18.208091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a52028-8de0-4ad6-a415-9a02a7d4575f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:18.210425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd54baa-3180-4d1f-9dae-3f67826ba0fe None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:18.214930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccabbc5-6f2b-4e49-9a22-5f5ee9ee40d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:18.215845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccabbc5-6f2b-4e49-9a22-5f5ee9ee40d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.216223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccabbc5-6f2b-4e49-9a22-5f5ee9ee40d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.216500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccabbc5-6f2b-4e49-9a22-5f5ee9ee40d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.218062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccabbc5-6f2b-4e49-9a22-5f5ee9ee40d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:18.221179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccabbc5-6f2b-4e49-9a22-5f5ee9ee40d0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:18.224305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23081d54-82a9-412d-84e7-498a0655183f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:18.225236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23081d54-82a9-412d-84e7-498a0655183f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.225504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23081d54-82a9-412d-84e7-498a0655183f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.226007 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23081d54-82a9-412d-84e7-498a0655183f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.238080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23081d54-82a9-412d-84e7-498a0655183f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:18.239649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23081d54-82a9-412d-84e7-498a0655183f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:18.252306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 991/1962] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:18.268709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 972/1963] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:18 2022] POST /identity/v3/projects => generated 407 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:18.271816 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23081d54-82a9-412d-84e7-498a0655183f service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:18.285463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23081d54-82a9-412d-84e7-498a0655183f service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:18.367384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769651dd-1846-45b1-817d-3a84a847a2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:18.368299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769651dd-1846-45b1-817d-3a84a847a2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.368731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769651dd-1846-45b1-817d-3a84a847a2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.369157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769651dd-1846-45b1-817d-3a84a847a2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.370117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769651dd-1846-45b1-817d-3a84a847a2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:18.372078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769651dd-1846-45b1-817d-3a84a847a2c7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:18.399410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 992/1964] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:18 2022] POST /identity/v3/users => generated 310 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:18.403273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccabbc5-6f2b-4e49-9a22-5f5ee9ee40d0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:18.417509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccabbc5-6f2b-4e49-9a22-5f5ee9ee40d0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:18.434440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfd52b-37d7-4ce3-afb7-bc12d02a8c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:18.443718 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfd52b-37d7-4ce3-afb7-bc12d02a8c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.444426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfd52b-37d7-4ce3-afb7-bc12d02a8c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.444713 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bfd52b-37d7-4ce3-afb7-bc12d02a8c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.446033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfd52b-37d7-4ce3-afb7-bc12d02a8c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:18.447633 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bfd52b-37d7-4ce3-afb7-bc12d02a8c94 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:18.462588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 973/1965] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:18.465773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769651dd-1846-45b1-817d-3a84a847a2c7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:18.478620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769651dd-1846-45b1-817d-3a84a847a2c7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:18.483665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518207b-b90a-4320-9b8d-47616079d8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:18.484466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518207b-b90a-4320-9b8d-47616079d8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.484734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518207b-b90a-4320-9b8d-47616079d8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.485130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f518207b-b90a-4320-9b8d-47616079d8bf admin admin] PATH_INFO: `/v3/projects/5cf2ab4d1f45428dbad8c149b16c452f/users/002b7f84dca04fbc928ebef5203a0498/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.492070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfd52b-37d7-4ce3-afb7-bc12d02a8c94 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:18.496512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28bfd52b-37d7-4ce3-afb7-bc12d02a8c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:18.510658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518207b-b90a-4320-9b8d-47616079d8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf2ab4d1f45428dbad8c149b16c452f, user_id=002b7f84dca04fbc928ebef5203a0498, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:18.510658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f518207b-b90a-4320-9b8d-47616079d8bf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:18.527351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bfd52b-37d7-4ce3-afb7-bc12d02a8c94 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:18.538008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 993/1966] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:18 2022] PUT /identity/v3/projects/5cf2ab4d1f45428dbad8c149b16c452f/users/002b7f84dca04fbc928ebef5203a0498/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:18.542554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518207b-b90a-4320-9b8d-47616079d8bf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:18.547958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:18.576918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:18.606096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:18.609710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.609710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.609710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] PATH_INFO: `/v3/projects/23e38dd0066145199fd481e34efec5b9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.610623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e38dd0066145199fd481e34efec5b9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:18.613160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:18.659621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f518207b-b90a-4320-9b8d-47616079d8bf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:18.675389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e38dd0066145199fd481e34efec5b9', 'request_id': 'req-0cb15c13-8f8b-4d24-a620-4c8412633419'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:18.683229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e38dd0066145199fd481e34efec5b9', 'request_id': 'req-0cb15c13-8f8b-4d24-a620-4c8412633419'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:18.700166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e38dd0066145199fd481e34efec5b9', 'request_id': 'req-0cb15c13-8f8b-4d24-a620-4c8412633419'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:18.717877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] The token cache is being invalidate because project 23e38dd0066145199fd481e34efec5b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:18.718252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:18.719370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0cb15c13-8f8b-4d24-a620-4c8412633419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:18.721472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 974/1967] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:18 2022] DELETE /identity/v3/projects/23e38dd0066145199fd481e34efec5b9 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:18.726057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d827-2992-4e57-b789-6f1ad788aed5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.726418 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d827-2992-4e57-b789-6f1ad788aed5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.726769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8d827-2992-4e57-b789-6f1ad788aed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.744229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18add5de-8dbf-4d3f-b41f-ed3fb45bc98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:18.744229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18add5de-8dbf-4d3f-b41f-ed3fb45bc98b admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.744491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18add5de-8dbf-4d3f-b41f-ed3fb45bc98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.744751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18add5de-8dbf-4d3f-b41f-ed3fb45bc98b admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.745707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18add5de-8dbf-4d3f-b41f-ed3fb45bc98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:18.747602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18add5de-8dbf-4d3f-b41f-ed3fb45bc98b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:18.756397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-80a8d827-2992-4e57-b789-6f1ad788aed5 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:18.757631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-80a8d827-2992-4e57-b789-6f1ad788aed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:18.763068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 994/1968] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:18 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:18.766383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18add5de-8dbf-4d3f-b41f-ed3fb45bc98b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:18.769415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-18add5de-8dbf-4d3f-b41f-ed3fb45bc98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:18.781704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 975/1969] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:18.788112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd1443-4daf-413d-bbfb-660d415f77c4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.788409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd1443-4daf-413d-bbfb-660d415f77c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.788660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd1443-4daf-413d-bbfb-660d415f77c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.806173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18add5de-8dbf-4d3f-b41f-ed3fb45bc98b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:18.815623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97ca2e-70fb-4451-93b8-0d47c0f222ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:18.824359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97ca2e-70fb-4451-93b8-0d47c0f222ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.824741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97ca2e-70fb-4451-93b8-0d47c0f222ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.826100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97ca2e-70fb-4451-93b8-0d47c0f222ff admin admin] PATH_INFO: `/v3/projects/5cf2ab4d1f45428dbad8c149b16c452f/users/002b7f84dca04fbc928ebef5203a0498/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.826859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-06fd1443-4daf-413d-bbfb-660d415f77c4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:18.828445 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97ca2e-70fb-4451-93b8-0d47c0f222ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cf2ab4d1f45428dbad8c149b16c452f, user_id=002b7f84dca04fbc928ebef5203a0498, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:18.829189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-06fd1443-4daf-413d-bbfb-660d415f77c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:18.831907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97ca2e-70fb-4451-93b8-0d47c0f222ff admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:18.838178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 976/1970] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:18.845086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd1443-4daf-413d-bbfb-660d415f77c4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:18.859913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd1443-4daf-413d-bbfb-660d415f77c4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:18.860369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 995/1971] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:18 2022] PUT /identity/v3/projects/5cf2ab4d1f45428dbad8c149b16c452f/users/002b7f84dca04fbc928ebef5203a0498/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:18.871718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacac300-59b6-40e0-b7d6-f336a2f5ee88 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.871997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacac300-59b6-40e0-b7d6-f336a2f5ee88 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.872813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacac300-59b6-40e0-b7d6-f336a2f5ee88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.922800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-eacac300-59b6-40e0-b7d6-f336a2f5ee88 None None] MFA Rules not processed for user `002b7f84dca04fbc928ebef5203a0498`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:18.925137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-eacac300-59b6-40e0-b7d6-f336a2f5ee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:18.986659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9309e2-7cd2-410c-97be-1d9f2a870dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:18.987921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9309e2-7cd2-410c-97be-1d9f2a870dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:18.988396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9309e2-7cd2-410c-97be-1d9f2a870dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:18.988681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9309e2-7cd2-410c-97be-1d9f2a870dcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:18.989407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9309e2-7cd2-410c-97be-1d9f2a870dcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:18.992629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9309e2-7cd2-410c-97be-1d9f2a870dcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:18.994758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9309e2-7cd2-410c-97be-1d9f2a870dcf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.013916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 977/1972] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:19.014874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9309e2-7cd2-410c-97be-1d9f2a870dcf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.016680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-da9309e2-7cd2-410c-97be-1d9f2a870dcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:19.034869 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9309e2-7cd2-410c-97be-1d9f2a870dcf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.042925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 996/1973] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:18 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:19.042925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacac300-59b6-40e0-b7d6-f336a2f5ee88 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.044277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.044991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.045272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.045555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] PATH_INFO: `/v3/projects/e9de6417d89141299453453981c5a4eb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.046993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9de6417d89141299453453981c5a4eb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.050298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.054020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacac300-59b6-40e0-b7d6-f336a2f5ee88 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.061031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa18e55-711d-4f6b-88eb-49ac1974ae81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.061869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa18e55-711d-4f6b-88eb-49ac1974ae81 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.062155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa18e55-711d-4f6b-88eb-49ac1974ae81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.062855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa18e55-711d-4f6b-88eb-49ac1974ae81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.064126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa18e55-711d-4f6b-88eb-49ac1974ae81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.066401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa18e55-711d-4f6b-88eb-49ac1974ae81 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.086158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:19.116848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 997/1974] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:19 2022] POST /identity/v3/projects => generated 363 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:19.118185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa18e55-711d-4f6b-88eb-49ac1974ae81 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.120166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8aa18e55-711d-4f6b-88eb-49ac1974ae81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:19.123836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9de6417d89141299453453981c5a4eb', 'request_id': 'req-7dfeaba5-855b-45d8-a458-d48271b8d02f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:19.128680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:19.132369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9de6417d89141299453453981c5a4eb', 'request_id': 'req-7dfeaba5-855b-45d8-a458-d48271b8d02f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:19.142636 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9de6417d89141299453453981c5a4eb', 'request_id': 'req-7dfeaba5-855b-45d8-a458-d48271b8d02f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:19.165716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] The token cache is being invalidate because project e9de6417d89141299453453981c5a4eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:19.166117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:19.167681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:19.173453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 978/1975] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:19 2022] DELETE /identity/v3/projects/e9de6417d89141299453453981c5a4eb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:19.173453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.173453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:19.216654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa18e55-711d-4f6b-88eb-49ac1974ae81 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.227085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329aee1-1ae4-4feb-abd9-4ccbebf1ebab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.227741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329aee1-1ae4-4feb-abd9-4ccbebf1ebab service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.228021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329aee1-1ae4-4feb-abd9-4ccbebf1ebab service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.228298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329aee1-1ae4-4feb-abd9-4ccbebf1ebab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.230365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5329aee1-1ae4-4feb-abd9-4ccbebf1ebab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:19.276938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfeaba5-855b-45d8-a458-d48271b8d02f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.334253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329aee1-1ae4-4feb-abd9-4ccbebf1ebab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.335723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329aee1-1ae4-4feb-abd9-4ccbebf1ebab service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.360445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965d62c-fb88-4c4d-8ab0-685a23ae3e79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.361187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965d62c-fb88-4c4d-8ab0-685a23ae3e79 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.361475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965d62c-fb88-4c4d-8ab0-685a23ae3e79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.361767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d965d62c-fb88-4c4d-8ab0-685a23ae3e79 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.362457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965d62c-fb88-4c4d-8ab0-685a23ae3e79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.364605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d965d62c-fb88-4c4d-8ab0-685a23ae3e79 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.365321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 998/1976] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:19 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:19.368255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44973214-1151-4b15-b07c-b33f7f28f5d2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.369031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44973214-1151-4b15-b07c-b33f7f28f5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.369031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44973214-1151-4b15-b07c-b33f7f28f5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.416887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 979/1977] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:19 2022] POST /identity/v3/users => generated 282 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:19.424614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965d62c-fb88-4c4d-8ab0-685a23ae3e79 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.431936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-44973214-1151-4b15-b07c-b33f7f28f5d2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:19.434341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-44973214-1151-4b15-b07c-b33f7f28f5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:19.447154 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d965d62c-fb88-4c4d-8ab0-685a23ae3e79 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.464076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 999/1978] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:19.466920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bee77e-7e8d-4d53-8a9e-cb59adca8314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.467818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bee77e-7e8d-4d53-8a9e-cb59adca8314 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.468179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bee77e-7e8d-4d53-8a9e-cb59adca8314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.468715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bee77e-7e8d-4d53-8a9e-cb59adca8314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.470008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bee77e-7e8d-4d53-8a9e-cb59adca8314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.473749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a5488-77b5-4b70-b491-f399c06ecc9d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.474281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a5488-77b5-4b70-b491-f399c06ecc9d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.474598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a5488-77b5-4b70-b491-f399c06ecc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.477723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bee77e-7e8d-4d53-8a9e-cb59adca8314 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.490219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 980/1979] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:19.494820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bee77e-7e8d-4d53-8a9e-cb59adca8314 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.508664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8d3a5488-77b5-4b70-b491-f399c06ecc9d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:19.509855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bee77e-7e8d-4d53-8a9e-cb59adca8314 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.510815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d3a5488-77b5-4b70-b491-f399c06ecc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:19.517003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa103b5c-83e5-4ce0-9a6d-86f7e3dd732a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.518323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa103b5c-83e5-4ce0-9a6d-86f7e3dd732a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.518835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa103b5c-83e5-4ce0-9a6d-86f7e3dd732a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.519428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa103b5c-83e5-4ce0-9a6d-86f7e3dd732a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.520797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa103b5c-83e5-4ce0-9a6d-86f7e3dd732a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.521101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1000/1980] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:19.526123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a5488-77b5-4b70-b491-f399c06ecc9d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.526981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa103b5c-83e5-4ce0-9a6d-86f7e3dd732a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.539903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a5488-77b5-4b70-b491-f399c06ecc9d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.540537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 981/1981] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:19.544387 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa103b5c-83e5-4ce0-9a6d-86f7e3dd732a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.546092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073eaef-78d2-490b-82be-94526389faf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.546861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073eaef-78d2-490b-82be-94526389faf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.547354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073eaef-78d2-490b-82be-94526389faf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.547634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073eaef-78d2-490b-82be-94526389faf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.548482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073eaef-78d2-490b-82be-94526389faf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:19.548783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073eaef-78d2-490b-82be-94526389faf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.550551 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073eaef-78d2-490b-82be-94526389faf2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.555334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa103b5c-83e5-4ce0-9a6d-86f7e3dd732a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.561702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4b002-0472-4bd4-afd0-98eb6436687b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.562621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4b002-0472-4bd4-afd0-98eb6436687b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.562932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4b002-0472-4bd4-afd0-98eb6436687b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.563267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a4b002-0472-4bd4-afd0-98eb6436687b admin admin] PATH_INFO: `/v3/projects/9cdde04ed4324908bd55bd38bcb36f9f/users/16b884a9f6a8444c8f06e0fd5ef909b2/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.565785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1001/1982] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:26:19.570253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073eaef-78d2-490b-82be-94526389faf2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.582186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073eaef-78d2-490b-82be-94526389faf2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.588924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4b002-0472-4bd4-afd0-98eb6436687b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cdde04ed4324908bd55bd38bcb36f9f, user_id=16b884a9f6a8444c8f06e0fd5ef909b2, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.591824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a4b002-0472-4bd4-afd0-98eb6436687b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.599016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92aea19-e120-49ee-9227-3e503f217aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.600109 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aea19-e120-49ee-9227-3e503f217aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.600492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aea19-e120-49ee-9227-3e503f217aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.600997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aea19-e120-49ee-9227-3e503f217aa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.603421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92aea19-e120-49ee-9227-3e503f217aa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.604904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92aea19-e120-49ee-9227-3e503f217aa6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.607573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:19.615261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 982/1983] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:19 2022] PUT /identity/v3/projects/9cdde04ed4324908bd55bd38bcb36f9f/users/16b884a9f6a8444c8f06e0fd5ef909b2/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:19.630405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4b002-0472-4bd4-afd0-98eb6436687b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.634177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94a4b002-0472-4bd4-afd0-98eb6436687b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:19.650569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1002/1984] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:19 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:19.651955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b786ed-6646-42c9-8b53-ff29a008ce95 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.652248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b786ed-6646-42c9-8b53-ff29a008ce95 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.652512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b786ed-6646-42c9-8b53-ff29a008ce95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.694784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-57b786ed-6646-42c9-8b53-ff29a008ce95 None None] MFA Rules not processed for user `16b884a9f6a8444c8f06e0fd5ef909b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:19.696527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-57b786ed-6646-42c9-8b53-ff29a008ce95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:19.706415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a4b002-0472-4bd4-afd0-98eb6436687b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.775249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e690b5-a5a0-4064-9c72-bc00c6968142 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.775793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e690b5-a5a0-4064-9c72-bc00c6968142 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.776214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e690b5-a5a0-4064-9c72-bc00c6968142 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.776544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e690b5-a5a0-4064-9c72-bc00c6968142 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.778622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-96e690b5-a5a0-4064-9c72-bc00c6968142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:19.781541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:19.800571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1003/1985] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:19 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:19.800571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b786ed-6646-42c9-8b53-ff29a008ce95 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.873433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e690b5-a5a0-4064-9c72-bc00c6968142 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.875490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e690b5-a5a0-4064-9c72-bc00c6968142 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.886164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 983/1986] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:19.886641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e690b5-a5a0-4064-9c72-bc00c6968142 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.887982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-96e690b5-a5a0-4064-9c72-bc00c6968142 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:19.899315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e690b5-a5a0-4064-9c72-bc00c6968142 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.900527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b786ed-6646-42c9-8b53-ff29a008ce95 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.903631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107cff9-abae-455f-8ea1-56330d2fdc1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.904536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107cff9-abae-455f-8ea1-56330d2fdc1e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.904915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107cff9-abae-455f-8ea1-56330d2fdc1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.905288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4107cff9-abae-455f-8ea1-56330d2fdc1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.906612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31845-7227-41bc-a9ff-4ab67ebe0af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.907405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31845-7227-41bc-a9ff-4ab67ebe0af2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.907865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31845-7227-41bc-a9ff-4ab67ebe0af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.908296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e31845-7227-41bc-a9ff-4ab67ebe0af2 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.909227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e31845-7227-41bc-a9ff-4ab67ebe0af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.911670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e31845-7227-41bc-a9ff-4ab67ebe0af2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.915497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107cff9-abae-455f-8ea1-56330d2fdc1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.917038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4107cff9-abae-455f-8ea1-56330d2fdc1e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.928604 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 984/1987] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:19 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:19.942194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1004/1988] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:19 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:19.946448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107cff9-abae-455f-8ea1-56330d2fdc1e service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:19.959110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4107cff9-abae-455f-8ea1-56330d2fdc1e service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:19.963919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774392d-38b2-4fc2-afb6-bc92ad8806f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:19.964547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774392d-38b2-4fc2-afb6-bc92ad8806f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:19.964956 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774392d-38b2-4fc2-afb6-bc92ad8806f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:19.965235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5774392d-38b2-4fc2-afb6-bc92ad8806f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:19.965962 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774392d-38b2-4fc2-afb6-bc92ad8806f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:19.967646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5774392d-38b2-4fc2-afb6-bc92ad8806f2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:19.983476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 985/1989] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:19.988268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31845-7227-41bc-a9ff-4ab67ebe0af2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:20.002041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e31845-7227-41bc-a9ff-4ab67ebe0af2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:20.020202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d241478-b8ab-4bcf-818b-9bb6e91ea2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:20.020963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d241478-b8ab-4bcf-818b-9bb6e91ea2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:20.021248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d241478-b8ab-4bcf-818b-9bb6e91ea2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:20.021526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d241478-b8ab-4bcf-818b-9bb6e91ea2b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:20.024263 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d241478-b8ab-4bcf-818b-9bb6e91ea2b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:20.025772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d241478-b8ab-4bcf-818b-9bb6e91ea2b2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:20.054333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1005/1990] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:19 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:20.058643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774392d-38b2-4fc2-afb6-bc92ad8806f2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:20.070334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5774392d-38b2-4fc2-afb6-bc92ad8806f2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:20.075198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2b280-37fe-4cb9-9b90-efece4cff049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:20.075928 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2b280-37fe-4cb9-9b90-efece4cff049 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:20.076434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2b280-37fe-4cb9-9b90-efece4cff049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:20.076620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2b280-37fe-4cb9-9b90-efece4cff049 admin admin] PATH_INFO: `/v3/projects/3916fb0ea9bd44ed93d36786e39d1340/users/5ce30abfd27249089142800a0a0e3bc6/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:20.096682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2b280-37fe-4cb9-9b90-efece4cff049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3916fb0ea9bd44ed93d36786e39d1340, user_id=5ce30abfd27249089142800a0a0e3bc6, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:20.099757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2b280-37fe-4cb9-9b90-efece4cff049 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:20.120881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 986/1991] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:20 2022] PUT /identity/v3/projects/3916fb0ea9bd44ed93d36786e39d1340/users/5ce30abfd27249089142800a0a0e3bc6/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:20.132797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18a8be-2b89-4d6c-93cb-b60456140c34 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:20.133085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18a8be-2b89-4d6c-93cb-b60456140c34 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:20.134294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18a8be-2b89-4d6c-93cb-b60456140c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:20.178253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1b18a8be-2b89-4d6c-93cb-b60456140c34 None None] MFA Rules not processed for user `5ce30abfd27249089142800a0a0e3bc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:20.179810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b18a8be-2b89-4d6c-93cb-b60456140c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:20.303310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1006/1992] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:20.368309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2b280-37fe-4cb9-9b90-efece4cff049 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:20.403295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:20.492374 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2b280-37fe-4cb9-9b90-efece4cff049 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:20.498048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155dfc19-15c0-4399-bd0a-1b40c85b806c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:20.499092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dfc19-15c0-4399-bd0a-1b40c85b806c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:20.499628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dfc19-15c0-4399-bd0a-1b40c85b806c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:20.500213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155dfc19-15c0-4399-bd0a-1b40c85b806c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:20.515851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155dfc19-15c0-4399-bd0a-1b40c85b806c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:20.517550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155dfc19-15c0-4399-bd0a-1b40c85b806c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:20.527784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 987/1993] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:20.551803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18a8be-2b89-4d6c-93cb-b60456140c34 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:20.584255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155dfc19-15c0-4399-bd0a-1b40c85b806c service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:20.585690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-155dfc19-15c0-4399-bd0a-1b40c85b806c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:20.628181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18a8be-2b89-4d6c-93cb-b60456140c34 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:20.634246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:20.634965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:20.635257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:20.635533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] PATH_INFO: `/v3/users/5ce30abfd27249089142800a0a0e3bc6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:20.637371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce30abfd27249089142800a0a0e3bc6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:20.639799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:20.650362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:20.673145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155dfc19-15c0-4399-bd0a-1b40c85b806c service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:20.678360 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:20.679759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:20.680177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:20.680522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] PATH_INFO: `/v3/users/8f1fcc1f5aea458db9c96e17478f390d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:20.682138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f1fcc1f5aea458db9c96e17478f390d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:20.684572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:20.728869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce30abfd27249089142800a0a0e3bc6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:20.729849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce30abfd27249089142800a0a0e3bc6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:20.751037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce30abfd27249089142800a0a0e3bc6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:20.752456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f1fcc1f5aea458db9c96e17478f390d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:20.753142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f1fcc1f5aea458db9c96e17478f390d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:20.761127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce30abfd27249089142800a0a0e3bc6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:20.770456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f1fcc1f5aea458db9c96e17478f390d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:20.773186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1c3d5e82-ab15-43e6-834a-061a0479b9ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce30abfd27249089142800a0a0e3bc6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:20.782063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1007/1994] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:20 2022] DELETE /identity/v3/users/5ce30abfd27249089142800a0a0e3bc6 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:20.786340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f1fcc1f5aea458db9c96e17478f390d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:20.786730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62606a05-357b-492d-99fe-1604c696847a None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:20.787138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62606a05-357b-492d-99fe-1604c696847a None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:20.787512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62606a05-357b-492d-99fe-1604c696847a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:20.801110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-67e92895-b035-417d-ac8d-7cf5a39ebe18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f1fcc1f5aea458db9c96e17478f390d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:20.809710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 988/1995] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:20 2022] DELETE /identity/v3/users/8f1fcc1f5aea458db9c96e17478f390d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:20.815265 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab4a6-8a58-45e9-b6fe-af699246b5a3 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:20.816023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab4a6-8a58-45e9-b6fe-af699246b5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:20.816537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dab4a6-8a58-45e9-b6fe-af699246b5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:20.816858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-62606a05-357b-492d-99fe-1604c696847a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:20.818545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-62606a05-357b-492d-99fe-1604c696847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:20.841088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-67dab4a6-8a58-45e9-b6fe-af699246b5a3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:20.843574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-67dab4a6-8a58-45e9-b6fe-af699246b5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:20.892307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:20.902775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1008/1996] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:20.912950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62606a05-357b-492d-99fe-1604c696847a None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:20.914611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-62606a05-357b-492d-99fe-1604c696847a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:20.928532 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 989/1997] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:20.940082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab4a6-8a58-45e9-b6fe-af699246b5a3 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:20.942305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-67dab4a6-8a58-45e9-b6fe-af699246b5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:20.992738 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62606a05-357b-492d-99fe-1604c696847a None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:20.997950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684d7c9-63d6-433d-a36f-98ae3309f986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:21.000039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d7c9-63d6-433d-a36f-98ae3309f986 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:21.000491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d7c9-63d6-433d-a36f-98ae3309f986 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:21.000792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684d7c9-63d6-433d-a36f-98ae3309f986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:21.018792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684d7c9-63d6-433d-a36f-98ae3309f986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:21.021787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684d7c9-63d6-433d-a36f-98ae3309f986 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:21.055175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dab4a6-8a58-45e9-b6fe-af699246b5a3 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:21.055900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1009/1998] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:21.061300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b0edd-fa75-40f7-8527-1aa8c9f36772 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:21.062129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0edd-fa75-40f7-8527-1aa8c9f36772 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:21.062501 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0edd-fa75-40f7-8527-1aa8c9f36772 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:21.062967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b0edd-fa75-40f7-8527-1aa8c9f36772 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:21.082455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b0edd-fa75-40f7-8527-1aa8c9f36772 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:21.083913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b0edd-fa75-40f7-8527-1aa8c9f36772 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:21.096908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 990/1999] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:21.620710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684d7c9-63d6-433d-a36f-98ae3309f986 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:21.624389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8684d7c9-63d6-433d-a36f-98ae3309f986 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:21.637366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684d7c9-63d6-433d-a36f-98ae3309f986 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:21.646032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:21.647304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:21.647304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:21.647538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] PATH_INFO: `/v3/projects/3916fb0ea9bd44ed93d36786e39d1340` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:21.648808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3916fb0ea9bd44ed93d36786e39d1340)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:21.651823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:21.673374 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b0edd-fa75-40f7-8527-1aa8c9f36772 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:21.675216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a6b0edd-fa75-40f7-8527-1aa8c9f36772 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:21.697589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b0edd-fa75-40f7-8527-1aa8c9f36772 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:21.705854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:21.707498 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:21.707789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:21.708086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] PATH_INFO: `/v3/projects/e7a428c7a603497b96c527f72aa33722` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:21.709392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a428c7a603497b96c527f72aa33722)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:21.711641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:21.738157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:21.784180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a428c7a603497b96c527f72aa33722', 'request_id': 'req-b2a5867a-3e64-4fcf-aa97-99875a7ba422'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:21.789870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3916fb0ea9bd44ed93d36786e39d1340', 'request_id': 'req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:21.798979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a428c7a603497b96c527f72aa33722', 'request_id': 'req-b2a5867a-3e64-4fcf-aa97-99875a7ba422'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:21.804423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3916fb0ea9bd44ed93d36786e39d1340', 'request_id': 'req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:21.811817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3916fb0ea9bd44ed93d36786e39d1340', 'request_id': 'req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:21.812567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a428c7a603497b96c527f72aa33722', 'request_id': 'req-b2a5867a-3e64-4fcf-aa97-99875a7ba422'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:21.827800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] The token cache is being invalidate because project 3916fb0ea9bd44ed93d36786e39d1340 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:21.828198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:21.828839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b8094b20-1216-48bf-bc2a-c5d9fad57fcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:21.831118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1010/2000] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:21 2022] DELETE /identity/v3/projects/3916fb0ea9bd44ed93d36786e39d1340 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:21.837963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59486c9c-005d-4a0e-9be7-d6941e07e1d5 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:21.837963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59486c9c-005d-4a0e-9be7-d6941e07e1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:21.837963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59486c9c-005d-4a0e-9be7-d6941e07e1d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:21.853917 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] The token cache is being invalidate because project e7a428c7a603497b96c527f72aa33722 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:21.854208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:21.858311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:21.860381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 991/2001] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:21 2022] DELETE /identity/v3/projects/e7a428c7a603497b96c527f72aa33722 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:21.860939 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-59486c9c-005d-4a0e-9be7-d6941e07e1d5 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:21.862316 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-59486c9c-005d-4a0e-9be7-d6941e07e1d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:21.864832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:21.866745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:21.940123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1011/2002] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:21.947739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b0a2a-2d43-4c75-bf4b-b3961e585472 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:21.948977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b0a2a-2d43-4c75-bf4b-b3961e585472 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:21.949534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b0a2a-2d43-4c75-bf4b-b3961e585472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:21.979570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d92b0a2a-2d43-4c75-bf4b-b3961e585472 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:21.981659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d92b0a2a-2d43-4c75-bf4b-b3961e585472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:21.994555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1012/2003] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:21.995896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b0a2a-2d43-4c75-bf4b-b3961e585472 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:21.998186 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5867a-3e64-4fcf-aa97-99875a7ba422 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:22.003603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:22.004209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.004491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.004827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] PATH_INFO: `/v3/users/35252113721d4748b6eb93a7ff1c4c23` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.007044 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35252113721d4748b6eb93a7ff1c4c23)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:22.009334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:22.010012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b0a2a-2d43-4c75-bf4b-b3961e585472 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:22.017004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579af2b-08ba-453d-9126-e4215320c1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:22.017681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579af2b-08ba-453d-9126-e4215320c1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.017992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579af2b-08ba-453d-9126-e4215320c1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.018260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579af2b-08ba-453d-9126-e4215320c1e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.019024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579af2b-08ba-453d-9126-e4215320c1e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:22.019380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579af2b-08ba-453d-9126-e4215320c1e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:22.021941 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579af2b-08ba-453d-9126-e4215320c1e9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:22.029984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:22.038110 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1013/2004] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:22.042290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579af2b-08ba-453d-9126-e4215320c1e9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:22.058575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579af2b-08ba-453d-9126-e4215320c1e9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:22.066902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a281dd7-cd2e-4e7e-92c8-0c3a3f8d6f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:22.067587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a281dd7-cd2e-4e7e-92c8-0c3a3f8d6f79 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.067975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a281dd7-cd2e-4e7e-92c8-0c3a3f8d6f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.068240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a281dd7-cd2e-4e7e-92c8-0c3a3f8d6f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.068979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a281dd7-cd2e-4e7e-92c8-0c3a3f8d6f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:22.071116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a281dd7-cd2e-4e7e-92c8-0c3a3f8d6f79 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:22.083768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35252113721d4748b6eb93a7ff1c4c23'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.084509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35252113721d4748b6eb93a7ff1c4c23'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.098935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35252113721d4748b6eb93a7ff1c4c23'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.104591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1014/2005] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:22 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:22.108850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a281dd7-cd2e-4e7e-92c8-0c3a3f8d6f79 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:22.109072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35252113721d4748b6eb93a7ff1c4c23'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.120371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35252113721d4748b6eb93a7ff1c4c23'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.130639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 992/2006] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:21 2022] DELETE /identity/v3/users/35252113721d4748b6eb93a7ff1c4c23 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:22.198577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a281dd7-cd2e-4e7e-92c8-0c3a3f8d6f79 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:22.219542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:22.300529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d2c4e-5fcf-4185-b4d7-44bf9699c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:22.302095 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d2c4e-5fcf-4185-b4d7-44bf9699c62d admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.302469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d2c4e-5fcf-4185-b4d7-44bf9699c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.303709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d2c4e-5fcf-4185-b4d7-44bf9699c62d admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.304272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d2c4e-5fcf-4185-b4d7-44bf9699c62d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:22.306400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d2c4e-5fcf-4185-b4d7-44bf9699c62d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:22.334204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1015/2007] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:22 2022] POST /identity/v3/users => generated 289 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:22.338038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:22.339466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:22.355005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee5497-a2fc-43fd-bc9c-89017f5f9049 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:22.360242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15418370-819d-4cc7-a54d-de0b43da37ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:22.360875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15418370-819d-4cc7-a54d-de0b43da37ad admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.361161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15418370-819d-4cc7-a54d-de0b43da37ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.361428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15418370-819d-4cc7-a54d-de0b43da37ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.362176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15418370-819d-4cc7-a54d-de0b43da37ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:22.363852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15418370-819d-4cc7-a54d-de0b43da37ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:22.378336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 993/2008] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:22.380028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d2c4e-5fcf-4185-b4d7-44bf9699c62d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:22.392816 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d2c4e-5fcf-4185-b4d7-44bf9699c62d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:22.398037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf11cb-995a-44c6-9e4b-dc4d6223468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:22.398764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf11cb-995a-44c6-9e4b-dc4d6223468f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.399106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf11cb-995a-44c6-9e4b-dc4d6223468f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.399439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf11cb-995a-44c6-9e4b-dc4d6223468f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.400208 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf11cb-995a-44c6-9e4b-dc4d6223468f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:22.401937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf11cb-995a-44c6-9e4b-dc4d6223468f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:22.412664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1016/2009] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:22.417021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15418370-819d-4cc7-a54d-de0b43da37ad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:22.432441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15418370-819d-4cc7-a54d-de0b43da37ad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:22.437680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ff71a2-54f4-4251-93b3-2f38cefb2cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:22.438461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff71a2-54f4-4251-93b3-2f38cefb2cac admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.438728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff71a2-54f4-4251-93b3-2f38cefb2cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.438994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ff71a2-54f4-4251-93b3-2f38cefb2cac admin admin] PATH_INFO: `/v3/projects/432fa11cbca14210b3f2b31f029d4414/users/bc1a0ebb775a46eeb21e6cbf2a599bbf/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.457022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff71a2-54f4-4251-93b3-2f38cefb2cac admin admin] RBAC: Authorizing `identity:create_grant(project_id=432fa11cbca14210b3f2b31f029d4414, user_id=bc1a0ebb775a46eeb21e6cbf2a599bbf, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:22.459057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf11cb-995a-44c6-9e4b-dc4d6223468f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:22.460309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ff71a2-54f4-4251-93b3-2f38cefb2cac admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:22.461439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-59bf11cb-995a-44c6-9e4b-dc4d6223468f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:22.482034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf11cb-995a-44c6-9e4b-dc4d6223468f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:22.486397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 994/2010] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:22 2022] PUT /identity/v3/projects/432fa11cbca14210b3f2b31f029d4414/users/bc1a0ebb775a46eeb21e6cbf2a599bbf/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:22.508361 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ee037-e73c-42f1-82b0-2e6b08783558 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.508831 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ee037-e73c-42f1-82b0-2e6b08783558 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.509316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ee037-e73c-42f1-82b0-2e6b08783558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.550057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e08ee037-e73c-42f1-82b0-2e6b08783558 None None] MFA Rules not processed for user `bc1a0ebb775a46eeb21e6cbf2a599bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:22.552386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e08ee037-e73c-42f1-82b0-2e6b08783558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:22.575040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:22.576125 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.576408 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.576687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] PATH_INFO: `/v3/projects/8c6123799f7443a38e861a0392f2ae93` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.578336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c6123799f7443a38e861a0392f2ae93)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:22.588186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:22.625124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 995/2011] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:22.637003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ee037-e73c-42f1-82b0-2e6b08783558 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:22.638557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e08ee037-e73c-42f1-82b0-2e6b08783558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:22.640473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c6123799f7443a38e861a0392f2ae93', 'request_id': 'req-031cb251-3d4c-4bb0-b999-f9f90e1404fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.650198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c6123799f7443a38e861a0392f2ae93', 'request_id': 'req-031cb251-3d4c-4bb0-b999-f9f90e1404fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.656253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c6123799f7443a38e861a0392f2ae93', 'request_id': 'req-031cb251-3d4c-4bb0-b999-f9f90e1404fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.670079 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] The token cache is being invalidate because project 8c6123799f7443a38e861a0392f2ae93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:22.670358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.670996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.672554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1017/2012] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:22 2022] DELETE /identity/v3/projects/8c6123799f7443a38e861a0392f2ae93 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:22.676517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:22.677903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:22.687795 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:22.700786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ee037-e73c-42f1-82b0-2e6b08783558 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:22.706746 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:22.758149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031cb251-3d4c-4bb0-b999-f9f90e1404fd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:22.758797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c29af5-b61b-4d74-b8dc-9d2743020fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:22.759538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c29af5-b61b-4d74-b8dc-9d2743020fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.759818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c29af5-b61b-4d74-b8dc-9d2743020fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.760091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c29af5-b61b-4d74-b8dc-9d2743020fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.762033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8c29af5-b61b-4d74-b8dc-9d2743020fa7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:22.763123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:22.763782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:22.764072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:22.764346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] PATH_INFO: `/v3/users/62637f2d901f4bb48d1811c5f05d13c9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:22.765619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] RBAC: Authorizing `identity:delete_user(user_id=62637f2d901f4bb48d1811c5f05d13c9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:22.767600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:22.821128 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62637f2d901f4bb48d1811c5f05d13c9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.821839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62637f2d901f4bb48d1811c5f05d13c9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.825959 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c29af5-b61b-4d74-b8dc-9d2743020fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:22.827455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c29af5-b61b-4d74-b8dc-9d2743020fa7 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:22.834590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62637f2d901f4bb48d1811c5f05d13c9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.836651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 996/2013] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:22.842552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62637f2d901f4bb48d1811c5f05d13c9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.852027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-85ef358e-9110-488f-ac65-3cab5ba0607c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62637f2d901f4bb48d1811c5f05d13c9'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:22.859139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1018/2014] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:22 2022] DELETE /identity/v3/users/62637f2d901f4bb48d1811c5f05d13c9 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:23.152821 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c29af5-b61b-4d74-b8dc-9d2743020fa7 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:23.168792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c29af5-b61b-4d74-b8dc-9d2743020fa7 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:23.234865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:23.235514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:23.235792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:23.236050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] PATH_INFO: `/v3/projects/2e846b7fa81a4524b311e58e215b9939` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:23.237210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e846b7fa81a4524b311e58e215b9939)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:23.239162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:23.296161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e846b7fa81a4524b311e58e215b9939', 'request_id': 'req-f147f812-17a9-4f16-b8bc-220ea8c9edc7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:23.301593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:23.303133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e846b7fa81a4524b311e58e215b9939', 'request_id': 'req-f147f812-17a9-4f16-b8bc-220ea8c9edc7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:23.312940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e846b7fa81a4524b311e58e215b9939', 'request_id': 'req-f147f812-17a9-4f16-b8bc-220ea8c9edc7'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:23.324553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] The token cache is being invalidate because project 2e846b7fa81a4524b311e58e215b9939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:23.324852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:23.325485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f147f812-17a9-4f16-b8bc-220ea8c9edc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:23.327153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 997/2015] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:23 2022] DELETE /identity/v3/projects/2e846b7fa81a4524b311e58e215b9939 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:23.333836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aaf92-b610-4eec-a801-8b92fc5f49ee None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:23.334299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aaf92-b610-4eec-a801-8b92fc5f49ee None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:23.334414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9aaf92-b610-4eec-a801-8b92fc5f49ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:23.355039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-6b9aaf92-b610-4eec-a801-8b92fc5f49ee None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:23.356194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6b9aaf92-b610-4eec-a801-8b92fc5f49ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:23.430301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1019/2016] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:23.439361 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ad63d-4c65-4e2f-8576-385242a76774 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:23.439648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ad63d-4c65-4e2f-8576-385242a76774 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:23.439923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2ad63d-4c65-4e2f-8576-385242a76774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:23.464336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7b2ad63d-4c65-4e2f-8576-385242a76774 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:23.465550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b2ad63d-4c65-4e2f-8576-385242a76774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:23.475654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 998/2017] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:23.480358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aaf92-b610-4eec-a801-8b92fc5f49ee None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:23.490282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9aaf92-b610-4eec-a801-8b92fc5f49ee None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:23.495322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c14955-cdeb-4417-9ffc-5ff71afb91f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:23.495965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c14955-cdeb-4417-9ffc-5ff71afb91f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:23.496240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c14955-cdeb-4417-9ffc-5ff71afb91f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:23.496508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c14955-cdeb-4417-9ffc-5ff71afb91f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:23.497199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c14955-cdeb-4417-9ffc-5ff71afb91f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:23.497495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c14955-cdeb-4417-9ffc-5ff71afb91f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:23.499127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c14955-cdeb-4417-9ffc-5ff71afb91f2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:23.510491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1020/2018] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:23.515163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ad63d-4c65-4e2f-8576-385242a76774 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:23.524882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2ad63d-4c65-4e2f-8576-385242a76774 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:23.530011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92646975-660a-4e13-aa53-fa8696a0f3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:23.530626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92646975-660a-4e13-aa53-fa8696a0f3e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:23.530901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92646975-660a-4e13-aa53-fa8696a0f3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:23.531169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92646975-660a-4e13-aa53-fa8696a0f3e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:23.531850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92646975-660a-4e13-aa53-fa8696a0f3e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:23.533770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92646975-660a-4e13-aa53-fa8696a0f3e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:23.566368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 999/2019] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:23 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:23.570784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c14955-cdeb-4417-9ffc-5ff71afb91f2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:23.582315 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c14955-cdeb-4417-9ffc-5ff71afb91f2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:23.614322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:23.644559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4226f-7b1b-4115-a512-a188a7e8c073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:23.645334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4226f-7b1b-4115-a512-a188a7e8c073 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:23.645611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4226f-7b1b-4115-a512-a188a7e8c073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:23.645913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4226f-7b1b-4115-a512-a188a7e8c073 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:23.646595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4226f-7b1b-4115-a512-a188a7e8c073 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:23.648516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4226f-7b1b-4115-a512-a188a7e8c073 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:23.676265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1021/2020] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:23 2022] POST /identity/v3/users => generated 281 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:23.678627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92646975-660a-4e13-aa53-fa8696a0f3e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:23.689338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92646975-660a-4e13-aa53-fa8696a0f3e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:23.695142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5717-5ca7-46ba-a9d8-a08716c01327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:23.695621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5717-5ca7-46ba-a9d8-a08716c01327 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:23.696021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5717-5ca7-46ba-a9d8-a08716c01327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:23.696303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5717-5ca7-46ba-a9d8-a08716c01327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:23.697000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb5717-5ca7-46ba-a9d8-a08716c01327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:23.698644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eb5717-5ca7-46ba-a9d8-a08716c01327 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:23.712190 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1000/2021] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:23.714291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4226f-7b1b-4115-a512-a188a7e8c073 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:23.725319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4226f-7b1b-4115-a512-a188a7e8c073 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:23.730174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f22cbf-c6d4-4291-ad45-56756cd60ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:23.730776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f22cbf-c6d4-4291-ad45-56756cd60ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:23.731056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f22cbf-c6d4-4291-ad45-56756cd60ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:23.731327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f22cbf-c6d4-4291-ad45-56756cd60ba5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:23.732025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f22cbf-c6d4-4291-ad45-56756cd60ba5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:23.733657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f22cbf-c6d4-4291-ad45-56756cd60ba5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:23.749601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1022/2022] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:23.751236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5717-5ca7-46ba-a9d8-a08716c01327 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:23.767877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5717-5ca7-46ba-a9d8-a08716c01327 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:23.772860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc412-9035-45bf-83b4-363e192b9154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:23.773578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc412-9035-45bf-83b4-363e192b9154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:23.774035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc412-9035-45bf-83b4-363e192b9154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:23.774315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc412-9035-45bf-83b4-363e192b9154 admin admin] PATH_INFO: `/v3/projects/1b38fb93602d4a348a5db3b295c01c5b/users/9c3f2649d3ff4e6bb4665a14abbec198/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:23.791409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddc412-9035-45bf-83b4-363e192b9154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b38fb93602d4a348a5db3b295c01c5b, user_id=9c3f2649d3ff4e6bb4665a14abbec198, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:23.794048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddc412-9035-45bf-83b4-363e192b9154 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:23.813042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1001/2023] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:23 2022] PUT /identity/v3/projects/1b38fb93602d4a348a5db3b295c01c5b/users/9c3f2649d3ff4e6bb4665a14abbec198/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:23.830143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db07f8-427f-47ce-a804-34aa70e9a7c3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:23.830428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db07f8-427f-47ce-a804-34aa70e9a7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:23.830700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db07f8-427f-47ce-a804-34aa70e9a7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:23.862466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-02db07f8-427f-47ce-a804-34aa70e9a7c3 None None] MFA Rules not processed for user `9c3f2649d3ff4e6bb4665a14abbec198`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:23.863613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-02db07f8-427f-47ce-a804-34aa70e9a7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:23.936774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1023/2024] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:23.974049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc412-9035-45bf-83b4-363e192b9154 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:23.976803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-adddc412-9035-45bf-83b4-363e192b9154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:24.036963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc412-9035-45bf-83b4-363e192b9154 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:24.041516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980bf7f7-e33a-4ec6-8c3e-f81a19cb19b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:24.042220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bf7f7-e33a-4ec6-8c3e-f81a19cb19b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:24.042504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bf7f7-e33a-4ec6-8c3e-f81a19cb19b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:24.042777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980bf7f7-e33a-4ec6-8c3e-f81a19cb19b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:24.052055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980bf7f7-e33a-4ec6-8c3e-f81a19cb19b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:24.053504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980bf7f7-e33a-4ec6-8c3e-f81a19cb19b1 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:24.062703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1002/2025] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:24.078122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db07f8-427f-47ce-a804-34aa70e9a7c3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:24.150237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db07f8-427f-47ce-a804-34aa70e9a7c3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:24.155408 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:24.156066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:24.156347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:24.156623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] PATH_INFO: `/v3/users/9c3f2649d3ff4e6bb4665a14abbec198` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:24.157858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c3f2649d3ff4e6bb4665a14abbec198)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:24.159805 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:24.205533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:24.213594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c3f2649d3ff4e6bb4665a14abbec198'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:24.214637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c3f2649d3ff4e6bb4665a14abbec198'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:24.229150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c3f2649d3ff4e6bb4665a14abbec198'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:24.235876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c3f2649d3ff4e6bb4665a14abbec198'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:24.245248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c3f2649d3ff4e6bb4665a14abbec198'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:24.251515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1024/2026] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:24 2022] DELETE /identity/v3/users/9c3f2649d3ff4e6bb4665a14abbec198 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:24.256185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e479932-0842-4e1b-985e-f4dd27e8b6bc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:24.256477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e479932-0842-4e1b-985e-f4dd27e8b6bc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:24.256788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e479932-0842-4e1b-985e-f4dd27e8b6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:24.276155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4e479932-0842-4e1b-985e-f4dd27e8b6bc None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:24.277370 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e479932-0842-4e1b-985e-f4dd27e8b6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:24.341213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1003/2027] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:24.353619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:24.353619 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:24.409441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c7302-c164-409f-837d-428c71da2a76 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:24.413196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49f02bd-13e1-4198-8733-af47c72c1395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:24.413838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f02bd-13e1-4198-8733-af47c72c1395 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:24.414119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f02bd-13e1-4198-8733-af47c72c1395 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:24.414389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49f02bd-13e1-4198-8733-af47c72c1395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:24.423354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f02bd-13e1-4198-8733-af47c72c1395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:24.424806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49f02bd-13e1-4198-8733-af47c72c1395 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:24.434994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1025/2028] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:24.730803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e479932-0842-4e1b-985e-f4dd27e8b6bc None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:24.732402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e479932-0842-4e1b-985e-f4dd27e8b6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:24.746173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e479932-0842-4e1b-985e-f4dd27e8b6bc None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:24.753938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:24.754768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:24.755173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:24.755457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] PATH_INFO: `/v3/projects/1b38fb93602d4a348a5db3b295c01c5b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:24.756772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b38fb93602d4a348a5db3b295c01c5b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:24.759161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:24.783279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:24.813313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b38fb93602d4a348a5db3b295c01c5b', 'request_id': 'req-35770e09-d51a-4db2-90f1-cabad8b7708f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:24.819105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b38fb93602d4a348a5db3b295c01c5b', 'request_id': 'req-35770e09-d51a-4db2-90f1-cabad8b7708f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:24.827532 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b38fb93602d4a348a5db3b295c01c5b', 'request_id': 'req-35770e09-d51a-4db2-90f1-cabad8b7708f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:24.839033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] The token cache is being invalidate because project 1b38fb93602d4a348a5db3b295c01c5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:24.839324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:24.840003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-35770e09-d51a-4db2-90f1-cabad8b7708f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:24.841554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1004/2029] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:24 2022] DELETE /identity/v3/projects/1b38fb93602d4a348a5db3b295c01c5b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:24.846932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afc512-00b6-49ce-b8c2-c28021e91c6c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:24.847237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afc512-00b6-49ce-b8c2-c28021e91c6c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:24.847532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afc512-00b6-49ce-b8c2-c28021e91c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:24.867044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-96afc512-00b6-49ce-b8c2-c28021e91c6c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:24.868260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-96afc512-00b6-49ce-b8c2-c28021e91c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:24.931431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1026/2030] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:24.939416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08618bea-536d-45b2-be17-ff8b99e2ca82 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:24.939712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08618bea-536d-45b2-be17-ff8b99e2ca82 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:24.939995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08618bea-536d-45b2-be17-ff8b99e2ca82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:24.959579 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-08618bea-536d-45b2-be17-ff8b99e2ca82 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:24.960792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-08618bea-536d-45b2-be17-ff8b99e2ca82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:24.969852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1005/2031] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:24.974214 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afc512-00b6-49ce-b8c2-c28021e91c6c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:24.984851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afc512-00b6-49ce-b8c2-c28021e91c6c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:24.990538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f8361-5184-4fe7-ac75-497c3164efdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:24.991158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f8361-5184-4fe7-ac75-497c3164efdd admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:24.991445 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f8361-5184-4fe7-ac75-497c3164efdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:24.991687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f8361-5184-4fe7-ac75-497c3164efdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:24.992516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f8361-5184-4fe7-ac75-497c3164efdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:24.992852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f8361-5184-4fe7-ac75-497c3164efdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:24.994530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f8361-5184-4fe7-ac75-497c3164efdd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.008815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1027/2032] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:25.015860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08618bea-536d-45b2-be17-ff8b99e2ca82 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:25.026019 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08618bea-536d-45b2-be17-ff8b99e2ca82 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:25.032669 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f403a9-bc59-42fc-bc1c-2e93d5da4d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:25.033722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f403a9-bc59-42fc-bc1c-2e93d5da4d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.034288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f403a9-bc59-42fc-bc1c-2e93d5da4d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.034799 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f403a9-bc59-42fc-bc1c-2e93d5da4d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.035978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f403a9-bc59-42fc-bc1c-2e93d5da4d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:25.039199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f403a9-bc59-42fc-bc1c-2e93d5da4d09 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.076678 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1006/2033] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:25 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:25.081099 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f8361-5184-4fe7-ac75-497c3164efdd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:25.092342 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f8361-5184-4fe7-ac75-497c3164efdd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:25.137254 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:25.150897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7d605-324b-40a1-b33f-61f503379a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:25.151523 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7d605-324b-40a1-b33f-61f503379a70 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.151800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7d605-324b-40a1-b33f-61f503379a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.152074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7d605-324b-40a1-b33f-61f503379a70 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.152748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7d605-324b-40a1-b33f-61f503379a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:25.154855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b7d605-324b-40a1-b33f-61f503379a70 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.178176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1028/2034] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:25 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:25.182101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f403a9-bc59-42fc-bc1c-2e93d5da4d09 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:25.192811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f403a9-bc59-42fc-bc1c-2e93d5da4d09 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:25.197424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130691a-b2c2-449e-826a-04eb68051111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:25.198141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130691a-b2c2-449e-826a-04eb68051111 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.198426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130691a-b2c2-449e-826a-04eb68051111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.198702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130691a-b2c2-449e-826a-04eb68051111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.199445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130691a-b2c2-449e-826a-04eb68051111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:25.201161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2130691a-b2c2-449e-826a-04eb68051111 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.211318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1007/2035] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:25.216977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7d605-324b-40a1-b33f-61f503379a70 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:25.229630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b7d605-324b-40a1-b33f-61f503379a70 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:25.236089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba936992-b9c1-453b-aeed-97e9269cec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:25.236743 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba936992-b9c1-453b-aeed-97e9269cec56 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.237191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba936992-b9c1-453b-aeed-97e9269cec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.237537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba936992-b9c1-453b-aeed-97e9269cec56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.238285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba936992-b9c1-453b-aeed-97e9269cec56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:25.239960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba936992-b9c1-453b-aeed-97e9269cec56 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.252401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1029/2036] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:25.257051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130691a-b2c2-449e-826a-04eb68051111 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:25.267429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130691a-b2c2-449e-826a-04eb68051111 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:25.272156 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c7047-366e-4dfe-ac93-0e0c95b91328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:25.272907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c7047-366e-4dfe-ac93-0e0c95b91328 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.273197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c7047-366e-4dfe-ac93-0e0c95b91328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.273466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c7047-366e-4dfe-ac93-0e0c95b91328 admin admin] PATH_INFO: `/v3/projects/8b172d22875140fc89b8b4fc749e218d/users/c16317f80360482a8021dc2a486c5c43/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.290044 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c7047-366e-4dfe-ac93-0e0c95b91328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b172d22875140fc89b8b4fc749e218d, user_id=c16317f80360482a8021dc2a486c5c43, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:25.292694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c7047-366e-4dfe-ac93-0e0c95b91328 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.306203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1008/2037] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:25 2022] PUT /identity/v3/projects/8b172d22875140fc89b8b4fc749e218d/users/c16317f80360482a8021dc2a486c5c43/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:25.317147 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f7258-fe00-4b5b-8328-5434cdf8ff37 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.317442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f7258-fe00-4b5b-8328-5434cdf8ff37 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.317918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f7258-fe00-4b5b-8328-5434cdf8ff37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.349765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cf8f7258-fe00-4b5b-8328-5434cdf8ff37 None None] MFA Rules not processed for user `c16317f80360482a8021dc2a486c5c43`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:25.351018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cf8f7258-fe00-4b5b-8328-5434cdf8ff37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:25.428403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1030/2038] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:25.429931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c7047-366e-4dfe-ac93-0e0c95b91328 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:25.507308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c7047-366e-4dfe-ac93-0e0c95b91328 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:25.512628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1b880-2696-4091-9d53-567674cb58e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:25.513358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1b880-2696-4091-9d53-567674cb58e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.513762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1b880-2696-4091-9d53-567674cb58e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.514046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1b880-2696-4091-9d53-567674cb58e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.514854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1b880-2696-4091-9d53-567674cb58e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:25.516828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1b880-2696-4091-9d53-567674cb58e9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.551730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1009/2039] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:25 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:25.552968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f7258-fe00-4b5b-8328-5434cdf8ff37 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:25.565341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f7258-fe00-4b5b-8328-5434cdf8ff37 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:25.629582 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09049fd9-6c0a-410f-a0f9-617f7f32b466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:25.630255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09049fd9-6c0a-410f-a0f9-617f7f32b466 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.630538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09049fd9-6c0a-410f-a0f9-617f7f32b466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.630820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09049fd9-6c0a-410f-a0f9-617f7f32b466 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.631499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09049fd9-6c0a-410f-a0f9-617f7f32b466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:25.633610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09049fd9-6c0a-410f-a0f9-617f7f32b466 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.668465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1031/2040] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:25 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:25.669891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1b880-2696-4091-9d53-567674cb58e9 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:25.685827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1b880-2696-4091-9d53-567674cb58e9 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:25.692923 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe65b88-885c-44d4-a750-bd059c0eec79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:25.693583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe65b88-885c-44d4-a750-bd059c0eec79 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.693900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe65b88-885c-44d4-a750-bd059c0eec79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.694173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe65b88-885c-44d4-a750-bd059c0eec79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.694894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe65b88-885c-44d4-a750-bd059c0eec79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:25.696519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe65b88-885c-44d4-a750-bd059c0eec79 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.714823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1010/2041] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:25.715483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09049fd9-6c0a-410f-a0f9-617f7f32b466 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:25.734295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09049fd9-6c0a-410f-a0f9-617f7f32b466 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:25.739375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316087f0-d390-42ba-a890-90a232ac0116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:25.740152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316087f0-d390-42ba-a890-90a232ac0116 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.740526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316087f0-d390-42ba-a890-90a232ac0116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.740899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316087f0-d390-42ba-a890-90a232ac0116 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.741715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316087f0-d390-42ba-a890-90a232ac0116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:25.743418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316087f0-d390-42ba-a890-90a232ac0116 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.754361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1032/2042] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:25.758678 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe65b88-885c-44d4-a750-bd059c0eec79 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:25.769211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe65b88-885c-44d4-a750-bd059c0eec79 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:25.773848 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a5e80-1833-47e4-98b6-b75aa755ec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:25.774511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a5e80-1833-47e4-98b6-b75aa755ec38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.774789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a5e80-1833-47e4-98b6-b75aa755ec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.775060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a5e80-1833-47e4-98b6-b75aa755ec38 admin admin] PATH_INFO: `/v3/projects/68d993f9663645daa3fabc12ad9065c2/users/474a1b0893ee4b52a6fd1ff21b8b0d88/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.789252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a5e80-1833-47e4-98b6-b75aa755ec38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68d993f9663645daa3fabc12ad9065c2, user_id=474a1b0893ee4b52a6fd1ff21b8b0d88, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:25.792073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a5e80-1833-47e4-98b6-b75aa755ec38 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:25.794274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:25.808472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1011/2043] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:25 2022] PUT /identity/v3/projects/68d993f9663645daa3fabc12ad9065c2/users/474a1b0893ee4b52a6fd1ff21b8b0d88/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:25.821546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be50b-7761-4e1f-a25d-cba91253a739 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:25.821876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be50b-7761-4e1f-a25d-cba91253a739 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:25.822160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61be50b-7761-4e1f-a25d-cba91253a739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:25.852104 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b61be50b-7761-4e1f-a25d-cba91253a739 None None] MFA Rules not processed for user `474a1b0893ee4b52a6fd1ff21b8b0d88`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:25.853316 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b61be50b-7761-4e1f-a25d-cba91253a739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:25.879718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:25.949679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1033/2044] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:25.952052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a5e80-1833-47e4-98b6-b75aa755ec38 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:26.043800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a5e80-1833-47e4-98b6-b75aa755ec38 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:26.050082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48802eb6-de30-4377-8ccb-edc419bf0bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:26.050723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802eb6-de30-4377-8ccb-edc419bf0bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:26.051001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802eb6-de30-4377-8ccb-edc419bf0bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:26.051306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802eb6-de30-4377-8ccb-edc419bf0bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:26.052132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48802eb6-de30-4377-8ccb-edc419bf0bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:26.054471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48802eb6-de30-4377-8ccb-edc419bf0bdf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:26.085984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1012/2045] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:25 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:26.089820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61be50b-7761-4e1f-a25d-cba91253a739 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:26.102731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61be50b-7761-4e1f-a25d-cba91253a739 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:26.171677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8ba3c-7986-470b-9416-31c4e2a50f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:26.172397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8ba3c-7986-470b-9416-31c4e2a50f11 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:26.172748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8ba3c-7986-470b-9416-31c4e2a50f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:26.173087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f8ba3c-7986-470b-9416-31c4e2a50f11 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:26.173869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8ba3c-7986-470b-9416-31c4e2a50f11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:26.176147 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f8ba3c-7986-470b-9416-31c4e2a50f11 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:26.201038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1034/2046] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:26 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:26.205217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48802eb6-de30-4377-8ccb-edc419bf0bdf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:26.218015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48802eb6-de30-4377-8ccb-edc419bf0bdf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:26.223639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b180b-6af0-4061-8809-7d0dd90bd753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:26.224267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b180b-6af0-4061-8809-7d0dd90bd753 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:26.224591 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b180b-6af0-4061-8809-7d0dd90bd753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:26.224872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b180b-6af0-4061-8809-7d0dd90bd753 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:26.225570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b180b-6af0-4061-8809-7d0dd90bd753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:26.227208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b180b-6af0-4061-8809-7d0dd90bd753 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:26.239917 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1013/2047] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:26.244852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8ba3c-7986-470b-9416-31c4e2a50f11 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:26.255334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f8ba3c-7986-470b-9416-31c4e2a50f11 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:26.261188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593fe63-7fe6-428e-9c0b-f7c1a8916446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:26.262133 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593fe63-7fe6-428e-9c0b-f7c1a8916446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:26.262441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593fe63-7fe6-428e-9c0b-f7c1a8916446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:26.262793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593fe63-7fe6-428e-9c0b-f7c1a8916446 admin admin] PATH_INFO: `/v3/projects/6691b682a75e42f69b8249e319aaa932/users/801b79a2cb764879a5dd3fa01df074de/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:26.281822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2593fe63-7fe6-428e-9c0b-f7c1a8916446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6691b682a75e42f69b8249e319aaa932, user_id=801b79a2cb764879a5dd3fa01df074de, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:26.284588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2593fe63-7fe6-428e-9c0b-f7c1a8916446 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:26.303368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1035/2048] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:26 2022] PUT /identity/v3/projects/6691b682a75e42f69b8249e319aaa932/users/801b79a2cb764879a5dd3fa01df074de/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:26.307194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b180b-6af0-4061-8809-7d0dd90bd753 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:26.318151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b180b-6af0-4061-8809-7d0dd90bd753 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:26.376367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7eb093-2495-4f57-9f3a-c590c6cb0e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:26.377123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7eb093-2495-4f57-9f3a-c590c6cb0e02 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:26.377467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7eb093-2495-4f57-9f3a-c590c6cb0e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:26.377841 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7eb093-2495-4f57-9f3a-c590c6cb0e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:26.378637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7eb093-2495-4f57-9f3a-c590c6cb0e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:26.380466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7eb093-2495-4f57-9f3a-c590c6cb0e02 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:26.391661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1014/2049] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:26.396948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593fe63-7fe6-428e-9c0b-f7c1a8916446 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:26.409327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593fe63-7fe6-428e-9c0b-f7c1a8916446 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:26.416168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc614b6-4e54-402d-a116-50d109e10ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:26.417125 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc614b6-4e54-402d-a116-50d109e10ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:26.417872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc614b6-4e54-402d-a116-50d109e10ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:26.418167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc614b6-4e54-402d-a116-50d109e10ff4 admin admin] PATH_INFO: `/v3/projects/6691b682a75e42f69b8249e319aaa932/users/801b79a2cb764879a5dd3fa01df074de/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:26.421828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc614b6-4e54-402d-a116-50d109e10ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6691b682a75e42f69b8249e319aaa932, user_id=801b79a2cb764879a5dd3fa01df074de, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:26.425540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc614b6-4e54-402d-a116-50d109e10ff4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:26.447007 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1036/2050] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:26 2022] PUT /identity/v3/projects/6691b682a75e42f69b8249e319aaa932/users/801b79a2cb764879a5dd3fa01df074de/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:26.458293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51421a-4f61-4133-be82-d85e134d1784 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:26.458779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51421a-4f61-4133-be82-d85e134d1784 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:26.459260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a51421a-4f61-4133-be82-d85e134d1784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:26.505774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4a51421a-4f61-4133-be82-d85e134d1784 None None] MFA Rules not processed for user `801b79a2cb764879a5dd3fa01df074de`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:26.507340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4a51421a-4f61-4133-be82-d85e134d1784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:26.575046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:26.612590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1015/2051] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:26.625380 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc614b6-4e54-402d-a116-50d109e10ff4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:26.627756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bdc614b6-4e54-402d-a116-50d109e10ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:26.693871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc614b6-4e54-402d-a116-50d109e10ff4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:26.697586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370aa77-7dca-47f8-9af0-d7d36ae7ac6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:26.698246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370aa77-7dca-47f8-9af0-d7d36ae7ac6f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:26.698519 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370aa77-7dca-47f8-9af0-d7d36ae7ac6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:26.698790 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370aa77-7dca-47f8-9af0-d7d36ae7ac6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:26.731403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:26.765573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370aa77-7dca-47f8-9af0-d7d36ae7ac6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:26.767287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370aa77-7dca-47f8-9af0-d7d36ae7ac6f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:26.777543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1037/2052] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:30.507989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51421a-4f61-4133-be82-d85e134d1784 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:30.518475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a51421a-4f61-4133-be82-d85e134d1784 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:30.522912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06516936-f9b3-43dc-a0d3-e0a1d5624d40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:30.523531 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06516936-f9b3-43dc-a0d3-e0a1d5624d40 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:30.523809 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06516936-f9b3-43dc-a0d3-e0a1d5624d40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:30.524093 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06516936-f9b3-43dc-a0d3-e0a1d5624d40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:30.584699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06516936-f9b3-43dc-a0d3-e0a1d5624d40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:30.586189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06516936-f9b3-43dc-a0d3-e0a1d5624d40 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:30.595920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1016/2053] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:30.760179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370aa77-7dca-47f8-9af0-d7d36ae7ac6f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:30.761795 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c370aa77-7dca-47f8-9af0-d7d36ae7ac6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:30.900931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370aa77-7dca-47f8-9af0-d7d36ae7ac6f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:30.908303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:30.909197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:30.909554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:30.910365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] PATH_INFO: `/v3/users/a4bf5a09529f478cba31c219471fc84f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:30.911788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4bf5a09529f478cba31c219471fc84f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:30.914946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:30.993032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4bf5a09529f478cba31c219471fc84f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:30.993777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4bf5a09529f478cba31c219471fc84f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.010065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4bf5a09529f478cba31c219471fc84f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.021068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4bf5a09529f478cba31c219471fc84f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.028693 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4bf5a09529f478cba31c219471fc84f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.037330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1038/2054] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:30 2022] DELETE /identity/v3/users/a4bf5a09529f478cba31c219471fc84f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:31.042348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7ebaa-3196-49d5-8268-7002226bca5c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:31.042790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7ebaa-3196-49d5-8268-7002226bca5c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:31.043140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb7ebaa-3196-49d5-8268-7002226bca5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:31.066266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0bb7ebaa-3196-49d5-8268-7002226bca5c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:31.067768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0bb7ebaa-3196-49d5-8268-7002226bca5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:31.131301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1017/2055] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:31.142741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:31.144313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:31.226890 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529cab3-b682-4932-880c-23ba876ec8f0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:31.238260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e24c0-4e79-44c3-bcee-da97db0f594f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:31.239662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e24c0-4e79-44c3-bcee-da97db0f594f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:31.240138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e24c0-4e79-44c3-bcee-da97db0f594f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:31.240425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e24c0-4e79-44c3-bcee-da97db0f594f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:31.248412 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:31.252853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e24c0-4e79-44c3-bcee-da97db0f594f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:31.254731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e24c0-4e79-44c3-bcee-da97db0f594f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:31.267700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1039/2056] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:31.300240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7ebaa-3196-49d5-8268-7002226bca5c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:31.301657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0bb7ebaa-3196-49d5-8268-7002226bca5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:31.378685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb7ebaa-3196-49d5-8268-7002226bca5c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:31.385574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf99643-f3b8-4cd1-801d-3ce55e62bb95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:31.386329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf99643-f3b8-4cd1-801d-3ce55e62bb95 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:31.386612 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf99643-f3b8-4cd1-801d-3ce55e62bb95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:31.386974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf99643-f3b8-4cd1-801d-3ce55e62bb95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:31.389153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dcf99643-f3b8-4cd1-801d-3ce55e62bb95 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:31.472048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:31.503839 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf99643-f3b8-4cd1-801d-3ce55e62bb95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:31.505282 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf99643-f3b8-4cd1-801d-3ce55e62bb95 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:31.515966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1018/2057] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:31.601386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e24c0-4e79-44c3-bcee-da97db0f594f service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:31.603005 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a27e24c0-4e79-44c3-bcee-da97db0f594f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:31.624266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e24c0-4e79-44c3-bcee-da97db0f594f service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:31.638530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:31.639189 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:31.639887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:31.640170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] PATH_INFO: `/v3/projects/11e20f95990b4ce2b2c06d3fcdd09b0c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:31.641355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=11e20f95990b4ce2b2c06d3fcdd09b0c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:31.643332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:31.693170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11e20f95990b4ce2b2c06d3fcdd09b0c', 'request_id': 'req-3399787a-75f8-41e0-9166-cff61e111fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.699653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11e20f95990b4ce2b2c06d3fcdd09b0c', 'request_id': 'req-3399787a-75f8-41e0-9166-cff61e111fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.710712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11e20f95990b4ce2b2c06d3fcdd09b0c', 'request_id': 'req-3399787a-75f8-41e0-9166-cff61e111fec'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.726778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] The token cache is being invalidate because project 11e20f95990b4ce2b2c06d3fcdd09b0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:31.727067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.727772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.729481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1040/2058] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:31 2022] DELETE /identity/v3/projects/11e20f95990b4ce2b2c06d3fcdd09b0c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:31.733939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf99643-f3b8-4cd1-801d-3ce55e62bb95 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:31.745415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf99643-f3b8-4cd1-801d-3ce55e62bb95 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:31.811651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:31.812329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:31.812608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:31.812880 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] PATH_INFO: `/v3/users/002b7f84dca04fbc928ebef5203a0498` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:31.814091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=002b7f84dca04fbc928ebef5203a0498)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:31.816023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:31.867046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '002b7f84dca04fbc928ebef5203a0498'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.867706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '002b7f84dca04fbc928ebef5203a0498'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.879699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '002b7f84dca04fbc928ebef5203a0498'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.886524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '002b7f84dca04fbc928ebef5203a0498'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.894974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-defba1cc-cea9-4ce9-a50c-608d2516dd70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '002b7f84dca04fbc928ebef5203a0498'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:31.901397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1019/2059] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:31 2022] DELETE /identity/v3/users/002b7f84dca04fbc928ebef5203a0498 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:32.236583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:32.238203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:32.251393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3399787a-75f8-41e0-9166-cff61e111fec admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:32.308975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:32.309637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:32.309946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:32.310262 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] PATH_INFO: `/v3/projects/5cf2ab4d1f45428dbad8c149b16c452f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:32.311460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cf2ab4d1f45428dbad8c149b16c452f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:32.313418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:32.361879 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cf2ab4d1f45428dbad8c149b16c452f', 'request_id': 'req-d93c960e-21f3-47e1-b5ef-ab807e181f15'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:32.368853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cf2ab4d1f45428dbad8c149b16c452f', 'request_id': 'req-d93c960e-21f3-47e1-b5ef-ab807e181f15'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:32.375061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cf2ab4d1f45428dbad8c149b16c452f', 'request_id': 'req-d93c960e-21f3-47e1-b5ef-ab807e181f15'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:32.391526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] The token cache is being invalidate because project 5cf2ab4d1f45428dbad8c149b16c452f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:32.391800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:32.392687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-d93c960e-21f3-47e1-b5ef-ab807e181f15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:32.395296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1041/2060] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:32 2022] DELETE /identity/v3/projects/5cf2ab4d1f45428dbad8c149b16c452f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:32.401867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2a847-8c8f-4bd8-ae12-d99fcdb32697 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:32.402284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2a847-8c8f-4bd8-ae12-d99fcdb32697 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:32.402647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be2a847-8c8f-4bd8-ae12-d99fcdb32697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:32.429142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0be2a847-8c8f-4bd8-ae12-d99fcdb32697 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:32.430335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0be2a847-8c8f-4bd8-ae12-d99fcdb32697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:32.483938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:32.495179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1020/2061] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:32.502935 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2e069-7965-4d33-96ff-e431f52f91e2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:32.503154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2e069-7965-4d33-96ff-e431f52f91e2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:32.503500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e2e069-7965-4d33-96ff-e431f52f91e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:32.521691 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:32.527839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-37e2e069-7965-4d33-96ff-e431f52f91e2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:32.529956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37e2e069-7965-4d33-96ff-e431f52f91e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:32.541926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1042/2062] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:32.546577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2a847-8c8f-4bd8-ae12-d99fcdb32697 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:32.556845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be2a847-8c8f-4bd8-ae12-d99fcdb32697 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:32.561611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556e387-9296-44e9-ae2f-ea5105515e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:32.562336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556e387-9296-44e9-ae2f-ea5105515e3e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:32.562620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556e387-9296-44e9-ae2f-ea5105515e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:32.562893 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7556e387-9296-44e9-ae2f-ea5105515e3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:32.563601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556e387-9296-44e9-ae2f-ea5105515e3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:32.563930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556e387-9296-44e9-ae2f-ea5105515e3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:32.565550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7556e387-9296-44e9-ae2f-ea5105515e3e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:32.577201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1021/2063] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:32.581887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2e069-7965-4d33-96ff-e431f52f91e2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:32.592035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e2e069-7965-4d33-96ff-e431f52f91e2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:32.598331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ee032-5f95-4376-a10d-db67f80cf893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:32.598945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ee032-5f95-4376-a10d-db67f80cf893 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:32.599224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ee032-5f95-4376-a10d-db67f80cf893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:32.599492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ee032-5f95-4376-a10d-db67f80cf893 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:32.600155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ee032-5f95-4376-a10d-db67f80cf893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:32.602118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ee032-5f95-4376-a10d-db67f80cf893 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:32.635156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1043/2064] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:32 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:32.639321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556e387-9296-44e9-ae2f-ea5105515e3e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:32.650333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7556e387-9296-44e9-ae2f-ea5105515e3e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:32.710963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375920e-6335-4c8c-a162-82bcaeb8699c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:32.711503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375920e-6335-4c8c-a162-82bcaeb8699c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:32.711783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375920e-6335-4c8c-a162-82bcaeb8699c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:32.712055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8375920e-6335-4c8c-a162-82bcaeb8699c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:32.712745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375920e-6335-4c8c-a162-82bcaeb8699c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:32.714716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8375920e-6335-4c8c-a162-82bcaeb8699c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:32.738625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1022/2065] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:32 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:32.743029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ee032-5f95-4376-a10d-db67f80cf893 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:32.754339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ee032-5f95-4376-a10d-db67f80cf893 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:32.759831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dda644-da92-4412-a72d-336286a75601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:32.760455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dda644-da92-4412-a72d-336286a75601 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:32.760734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dda644-da92-4412-a72d-336286a75601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:32.761006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dda644-da92-4412-a72d-336286a75601 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:32.761710 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dda644-da92-4412-a72d-336286a75601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:32.763352 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dda644-da92-4412-a72d-336286a75601 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:32.775052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1044/2066] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:32.779448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375920e-6335-4c8c-a162-82bcaeb8699c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:32.790411 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8375920e-6335-4c8c-a162-82bcaeb8699c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:32.795561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058ed375-8b3b-4b96-8443-88c2f61e3e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:32.796188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ed375-8b3b-4b96-8443-88c2f61e3e20 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:32.796470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ed375-8b3b-4b96-8443-88c2f61e3e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:32.796749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058ed375-8b3b-4b96-8443-88c2f61e3e20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:32.797456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ed375-8b3b-4b96-8443-88c2f61e3e20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:32.799097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058ed375-8b3b-4b96-8443-88c2f61e3e20 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:32.811927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1023/2067] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:32.817207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dda644-da92-4412-a72d-336286a75601 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:32.831756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dda644-da92-4412-a72d-336286a75601 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:32.839266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac202e-d208-4575-80c6-2da0fcc82738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:32.839960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac202e-d208-4575-80c6-2da0fcc82738 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:32.840231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac202e-d208-4575-80c6-2da0fcc82738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:32.840517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fac202e-d208-4575-80c6-2da0fcc82738 admin admin] PATH_INFO: `/v3/projects/b359fac4f21a49caab1d25b79227280f/users/51225363a61d4c7a9d25883d55a1887d/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:32.857609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac202e-d208-4575-80c6-2da0fcc82738 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b359fac4f21a49caab1d25b79227280f, user_id=51225363a61d4c7a9d25883d55a1887d, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:32.860369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fac202e-d208-4575-80c6-2da0fcc82738 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:32.874773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1045/2068] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:32 2022] PUT /identity/v3/projects/b359fac4f21a49caab1d25b79227280f/users/51225363a61d4c7a9d25883d55a1887d/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:32.889046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d66900-673c-43a7-aa7a-bea12270dce0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:32.889348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d66900-673c-43a7-aa7a-bea12270dce0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:32.889625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d66900-673c-43a7-aa7a-bea12270dce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:32.919398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a1d66900-673c-43a7-aa7a-bea12270dce0 None None] MFA Rules not processed for user `51225363a61d4c7a9d25883d55a1887d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:32.920916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1d66900-673c-43a7-aa7a-bea12270dce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:32.996090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1024/2069] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:33.000772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac202e-d208-4575-80c6-2da0fcc82738 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:33.065111 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fac202e-d208-4575-80c6-2da0fcc82738 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:33.070630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9265f07-417a-47d9-aac5-2a70b70a37ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:33.072968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9265f07-417a-47d9-aac5-2a70b70a37ba admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:33.073326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9265f07-417a-47d9-aac5-2a70b70a37ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:33.073671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9265f07-417a-47d9-aac5-2a70b70a37ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:33.074909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9265f07-417a-47d9-aac5-2a70b70a37ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:33.076667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9265f07-417a-47d9-aac5-2a70b70a37ba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:33.111801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1046/2070] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:32 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:33.113297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d66900-673c-43a7-aa7a-bea12270dce0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:33.124666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d66900-673c-43a7-aa7a-bea12270dce0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:33.176016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:33.188614 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999ee11-0977-409f-a5c5-f774461335df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:33.189457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999ee11-0977-409f-a5c5-f774461335df admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:33.189888 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999ee11-0977-409f-a5c5-f774461335df admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:33.190226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3999ee11-0977-409f-a5c5-f774461335df admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:33.191017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999ee11-0977-409f-a5c5-f774461335df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:33.193064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3999ee11-0977-409f-a5c5-f774461335df admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:33.223910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1025/2071] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:33 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:33.228072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9265f07-417a-47d9-aac5-2a70b70a37ba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:33.239008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9265f07-417a-47d9-aac5-2a70b70a37ba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:33.244052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a20654-9596-4547-8275-2d444462a346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:33.244849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a20654-9596-4547-8275-2d444462a346 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:33.245336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a20654-9596-4547-8275-2d444462a346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:33.245638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a20654-9596-4547-8275-2d444462a346 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:33.246504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a20654-9596-4547-8275-2d444462a346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:33.248186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a20654-9596-4547-8275-2d444462a346 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:33.262644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1047/2072] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:33.263990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999ee11-0977-409f-a5c5-f774461335df admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:33.274195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3999ee11-0977-409f-a5c5-f774461335df admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:33.278827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5e097-3a8e-4f13-b026-fcd3f5c1f820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:33.279567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e097-3a8e-4f13-b026-fcd3f5c1f820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:33.279903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e097-3a8e-4f13-b026-fcd3f5c1f820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:33.280178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e097-3a8e-4f13-b026-fcd3f5c1f820 admin admin] PATH_INFO: `/v3/projects/486a390412ea4fce971cb760f72445a9/users/dae015303a214f33aef4bd9b91466ffe/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:33.295680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5e097-3a8e-4f13-b026-fcd3f5c1f820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486a390412ea4fce971cb760f72445a9, user_id=dae015303a214f33aef4bd9b91466ffe, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:33.298394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed5e097-3a8e-4f13-b026-fcd3f5c1f820 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:33.312550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1026/2073] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:33 2022] PUT /identity/v3/projects/486a390412ea4fce971cb760f72445a9/users/dae015303a214f33aef4bd9b91466ffe/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:33.316444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a20654-9596-4547-8275-2d444462a346 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:33.326328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a20654-9596-4547-8275-2d444462a346 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:33.388134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed9c9c1-60d5-4da6-bdc0-c6d0d08c8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:33.388944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9c9c1-60d5-4da6-bdc0-c6d0d08c8204 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:33.389221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9c9c1-60d5-4da6-bdc0-c6d0d08c8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:33.389494 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed9c9c1-60d5-4da6-bdc0-c6d0d08c8204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:33.390214 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9c9c1-60d5-4da6-bdc0-c6d0d08c8204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:33.391810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed9c9c1-60d5-4da6-bdc0-c6d0d08c8204 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:33.401498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1048/2074] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:33.406472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5e097-3a8e-4f13-b026-fcd3f5c1f820 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:33.423280 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5e097-3a8e-4f13-b026-fcd3f5c1f820 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:33.429145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15727f9-40de-4f7b-b5d7-cb929d8e0ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:33.430073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15727f9-40de-4f7b-b5d7-cb929d8e0ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:33.430358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15727f9-40de-4f7b-b5d7-cb929d8e0ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:33.430730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15727f9-40de-4f7b-b5d7-cb929d8e0ff2 admin admin] PATH_INFO: `/v3/projects/486a390412ea4fce971cb760f72445a9/users/dae015303a214f33aef4bd9b91466ffe/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:33.433456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15727f9-40de-4f7b-b5d7-cb929d8e0ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=486a390412ea4fce971cb760f72445a9, user_id=dae015303a214f33aef4bd9b91466ffe, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:33.436644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15727f9-40de-4f7b-b5d7-cb929d8e0ff2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:33.448876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1027/2075] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:33 2022] PUT /identity/v3/projects/486a390412ea4fce971cb760f72445a9/users/dae015303a214f33aef4bd9b91466ffe/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:33.460535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bc4fd-5164-480c-82d7-f8d803045218 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:33.460822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bc4fd-5164-480c-82d7-f8d803045218 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:33.461103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1bc4fd-5164-480c-82d7-f8d803045218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:33.487509 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:33.491244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9a1bc4fd-5164-480c-82d7-f8d803045218 None None] MFA Rules not processed for user `dae015303a214f33aef4bd9b91466ffe`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:33.493164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a1bc4fd-5164-480c-82d7-f8d803045218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:33.572845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1049/2076] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:33.585236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15727f9-40de-4f7b-b5d7-cb929d8e0ff2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:33.587343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b15727f9-40de-4f7b-b5d7-cb929d8e0ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:33.651928 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15727f9-40de-4f7b-b5d7-cb929d8e0ff2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:33.657305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fefae-1b61-4942-9672-3d9d6ec864c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:33.657975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fefae-1b61-4942-9672-3d9d6ec864c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:33.658264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fefae-1b61-4942-9672-3d9d6ec864c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:33.658528 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905fefae-1b61-4942-9672-3d9d6ec864c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:33.670302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fefae-1b61-4942-9672-3d9d6ec864c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:33.671748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905fefae-1b61-4942-9672-3d9d6ec864c5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:33.681762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1028/2077] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:33.762626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bc4fd-5164-480c-82d7-f8d803045218 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:33.764235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9a1bc4fd-5164-480c-82d7-f8d803045218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:33.776587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1bc4fd-5164-480c-82d7-f8d803045218 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:33.780443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0982f0df-0e59-4445-8c3d-13ba8d69de97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:33.781090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982f0df-0e59-4445-8c3d-13ba8d69de97 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:33.781370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982f0df-0e59-4445-8c3d-13ba8d69de97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:33.781642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0982f0df-0e59-4445-8c3d-13ba8d69de97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:33.846418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982f0df-0e59-4445-8c3d-13ba8d69de97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:33.847796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0982f0df-0e59-4445-8c3d-13ba8d69de97 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:33.857047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1050/2078] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:33 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:34.133416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fefae-1b61-4942-9672-3d9d6ec864c5 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:34.202333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905fefae-1b61-4942-9672-3d9d6ec864c5 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:34.207300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:34.207939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:34.208217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:34.208488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] PATH_INFO: `/v3/users/51225363a61d4c7a9d25883d55a1887d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:34.209671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51225363a61d4c7a9d25883d55a1887d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:34.211708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:34.261110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51225363a61d4c7a9d25883d55a1887d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:34.261882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51225363a61d4c7a9d25883d55a1887d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:34.263892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:34.276668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51225363a61d4c7a9d25883d55a1887d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:34.284544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51225363a61d4c7a9d25883d55a1887d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:34.292675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51225363a61d4c7a9d25883d55a1887d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:34.299286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1029/2079] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:34 2022] DELETE /identity/v3/users/51225363a61d4c7a9d25883d55a1887d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:34.303960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c644cc-25c3-4a0c-8947-1962ba28478f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:34.304574 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c644cc-25c3-4a0c-8947-1962ba28478f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:34.304678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c644cc-25c3-4a0c-8947-1962ba28478f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:34.326141 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-04c644cc-25c3-4a0c-8947-1962ba28478f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:34.327345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-04c644cc-25c3-4a0c-8947-1962ba28478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:34.392453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1051/2080] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:34.402126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:34.403607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:34.481380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b5bb80-fb00-4b5a-a0ce-701b3c109189 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:34.485180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81435a-31b8-41ea-9fe3-62c8831add57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:34.486259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81435a-31b8-41ea-9fe3-62c8831add57 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:34.486259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81435a-31b8-41ea-9fe3-62c8831add57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:34.486488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81435a-31b8-41ea-9fe3-62c8831add57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:34.496421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81435a-31b8-41ea-9fe3-62c8831add57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:34.498051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81435a-31b8-41ea-9fe3-62c8831add57 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:34.508723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1030/2081] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:34.884195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c644cc-25c3-4a0c-8947-1962ba28478f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:34.885902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-04c644cc-25c3-4a0c-8947-1962ba28478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:34.900520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c644cc-25c3-4a0c-8947-1962ba28478f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:34.906340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:34.907057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:34.907359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:34.907657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] PATH_INFO: `/v3/projects/b359fac4f21a49caab1d25b79227280f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:34.909035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b359fac4f21a49caab1d25b79227280f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:34.911008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:34.949471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:34.963230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b359fac4f21a49caab1d25b79227280f', 'request_id': 'req-216ef35b-bb53-4cc4-9680-81ac59469624'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:34.969950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b359fac4f21a49caab1d25b79227280f', 'request_id': 'req-216ef35b-bb53-4cc4-9680-81ac59469624'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:34.975987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b359fac4f21a49caab1d25b79227280f', 'request_id': 'req-216ef35b-bb53-4cc4-9680-81ac59469624'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:34.989775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] The token cache is being invalidate because project b359fac4f21a49caab1d25b79227280f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:34.990074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:34.990687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:34.992243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1052/2082] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:34 2022] DELETE /identity/v3/projects/b359fac4f21a49caab1d25b79227280f => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:34.996563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81435a-31b8-41ea-9fe3-62c8831add57 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:35.006820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81435a-31b8-41ea-9fe3-62c8831add57 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:35.070727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:35.071377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:35.071663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:35.071919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] PATH_INFO: `/v3/users/dae015303a214f33aef4bd9b91466ffe` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:35.073124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dae015303a214f33aef4bd9b91466ffe)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:35.075108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:35.128530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dae015303a214f33aef4bd9b91466ffe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:35.129198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dae015303a214f33aef4bd9b91466ffe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:35.143576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dae015303a214f33aef4bd9b91466ffe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:35.149285 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:35.151368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dae015303a214f33aef4bd9b91466ffe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:35.160103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c32b9a2c-8b64-4b49-9220-c62ceb28626a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dae015303a214f33aef4bd9b91466ffe'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:35.167099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1031/2083] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:34 2022] DELETE /identity/v3/users/dae015303a214f33aef4bd9b91466ffe => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:35.799954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:35.801359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:35.819416 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ef35b-bb53-4cc4-9680-81ac59469624 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:35.932532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:35.933642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:35.933952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:35.934251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] PATH_INFO: `/v3/projects/486a390412ea4fce971cb760f72445a9` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:35.937056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=486a390412ea4fce971cb760f72445a9)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:35.939392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:36.019388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '486a390412ea4fce971cb760f72445a9', 'request_id': 'req-5ddac279-6abc-4db4-9f76-a5ff7fd20318'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:36.034918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '486a390412ea4fce971cb760f72445a9', 'request_id': 'req-5ddac279-6abc-4db4-9f76-a5ff7fd20318'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:36.050795 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '486a390412ea4fce971cb760f72445a9', 'request_id': 'req-5ddac279-6abc-4db4-9f76-a5ff7fd20318'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:36.090913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] The token cache is being invalidate because project 486a390412ea4fce971cb760f72445a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:36.090913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:36.091604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5ddac279-6abc-4db4-9f76-a5ff7fd20318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:36.097160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1053/2084] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:35 2022] DELETE /identity/v3/projects/486a390412ea4fce971cb760f72445a9 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:36.102951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19c9a6-ca84-475f-83ad-ee901e6f2641 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:36.103557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19c9a6-ca84-475f-83ad-ee901e6f2641 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:36.103557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19c9a6-ca84-475f-83ad-ee901e6f2641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:36.159175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4e19c9a6-ca84-475f-83ad-ee901e6f2641 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:36.160570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e19c9a6-ca84-475f-83ad-ee901e6f2641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:36.259994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1032/2085] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:36.268965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d8e43-0a1e-4a5b-bf59-c1bade282b25 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:36.269268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d8e43-0a1e-4a5b-bf59-c1bade282b25 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:36.269533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d8e43-0a1e-4a5b-bf59-c1bade282b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:36.290623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c6d8e43-0a1e-4a5b-bf59-c1bade282b25 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:36.291818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c6d8e43-0a1e-4a5b-bf59-c1bade282b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:36.304015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1054/2086] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:36.309843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19c9a6-ca84-475f-83ad-ee901e6f2641 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:36.320317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19c9a6-ca84-475f-83ad-ee901e6f2641 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:36.325616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7460491-6c54-4cbc-9842-be0a9dd202a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:36.326310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460491-6c54-4cbc-9842-be0a9dd202a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:36.326590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460491-6c54-4cbc-9842-be0a9dd202a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:36.326865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460491-6c54-4cbc-9842-be0a9dd202a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:36.327568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7460491-6c54-4cbc-9842-be0a9dd202a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:36.327870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7460491-6c54-4cbc-9842-be0a9dd202a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:36.329563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7460491-6c54-4cbc-9842-be0a9dd202a6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:36.349231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d8e43-0a1e-4a5b-bf59-c1bade282b25 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:36.353874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1033/2087] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:36.358883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d8e43-0a1e-4a5b-bf59-c1bade282b25 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:36.363667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4fb5a-7c19-417b-8cfb-a99c263117c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:36.364458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4fb5a-7c19-417b-8cfb-a99c263117c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:36.364697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4fb5a-7c19-417b-8cfb-a99c263117c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:36.364984 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4fb5a-7c19-417b-8cfb-a99c263117c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:36.365766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4fb5a-7c19-417b-8cfb-a99c263117c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:36.367799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4fb5a-7c19-417b-8cfb-a99c263117c0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:36.400005 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1055/2088] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:36 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:36.404787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7460491-6c54-4cbc-9842-be0a9dd202a6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:36.416025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7460491-6c54-4cbc-9842-be0a9dd202a6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:36.497199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da4f38-bf84-4c8b-bc09-977d5ee315d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:36.497963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da4f38-bf84-4c8b-bc09-977d5ee315d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:36.498268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da4f38-bf84-4c8b-bc09-977d5ee315d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:36.498476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da4f38-bf84-4c8b-bc09-977d5ee315d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:36.499156 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da4f38-bf84-4c8b-bc09-977d5ee315d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:36.501312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da4f38-bf84-4c8b-bc09-977d5ee315d1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:36.527537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1034/2089] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:36 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:36.531657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4fb5a-7c19-417b-8cfb-a99c263117c0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:36.546601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4fb5a-7c19-417b-8cfb-a99c263117c0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:36.553369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb0799-d789-4ecb-a410-d5d6ee7befc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:36.554442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb0799-d789-4ecb-a410-d5d6ee7befc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:36.554797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb0799-d789-4ecb-a410-d5d6ee7befc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:36.556841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bb0799-d789-4ecb-a410-d5d6ee7befc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:36.556841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb0799-d789-4ecb-a410-d5d6ee7befc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:36.557826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bb0799-d789-4ecb-a410-d5d6ee7befc8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:36.570450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1056/2090] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:36.574752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da4f38-bf84-4c8b-bc09-977d5ee315d1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:36.585283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da4f38-bf84-4c8b-bc09-977d5ee315d1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:36.590475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be65541-de4f-4f51-919b-c46b2e9be26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:36.591192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be65541-de4f-4f51-919b-c46b2e9be26b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:36.591615 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be65541-de4f-4f51-919b-c46b2e9be26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:36.591967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be65541-de4f-4f51-919b-c46b2e9be26b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:36.592750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be65541-de4f-4f51-919b-c46b2e9be26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:36.594782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be65541-de4f-4f51-919b-c46b2e9be26b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:36.607290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1035/2091] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:36.611637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb0799-d789-4ecb-a410-d5d6ee7befc8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:36.624601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bb0799-d789-4ecb-a410-d5d6ee7befc8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:36.630561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da751f29-464a-4749-9f55-4e11e7e54f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:36.631254 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da751f29-464a-4749-9f55-4e11e7e54f9e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:36.631538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da751f29-464a-4749-9f55-4e11e7e54f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:36.631817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da751f29-464a-4749-9f55-4e11e7e54f9e admin admin] PATH_INFO: `/v3/projects/72b2929a4d1749539a930c8847362f68/users/65139c98d95b4b6493e6136bf74a619d/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:36.643610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:36.653374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da751f29-464a-4749-9f55-4e11e7e54f9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=72b2929a4d1749539a930c8847362f68, user_id=65139c98d95b4b6493e6136bf74a619d, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:36.658081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da751f29-464a-4749-9f55-4e11e7e54f9e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:36.675736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1057/2092] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:36 2022] PUT /identity/v3/projects/72b2929a4d1749539a930c8847362f68/users/65139c98d95b4b6493e6136bf74a619d/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:36.688808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306dd38-7d82-45ea-8ff0-55f74ac67de5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:36.688985 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306dd38-7d82-45ea-8ff0-55f74ac67de5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:36.689246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306dd38-7d82-45ea-8ff0-55f74ac67de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:36.726107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-1306dd38-7d82-45ea-8ff0-55f74ac67de5 None None] MFA Rules not processed for user `65139c98d95b4b6493e6136bf74a619d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:36.727392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1306dd38-7d82-45ea-8ff0-55f74ac67de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:36.782400 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:36.812704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1036/2093] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:36.829462 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da751f29-464a-4749-9f55-4e11e7e54f9e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:36.831521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-da751f29-464a-4749-9f55-4e11e7e54f9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:36.901341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da751f29-464a-4749-9f55-4e11e7e54f9e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:36.905121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc6dc6-4921-4984-8bd3-aa3d0ef1d2f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:36.905763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc6dc6-4921-4984-8bd3-aa3d0ef1d2f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:36.906060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc6dc6-4921-4984-8bd3-aa3d0ef1d2f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:36.906338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc6dc6-4921-4984-8bd3-aa3d0ef1d2f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:36.914824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc6dc6-4921-4984-8bd3-aa3d0ef1d2f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:36.916267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc6dc6-4921-4984-8bd3-aa3d0ef1d2f3 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:36.926124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1058/2094] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:41.708865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306dd38-7d82-45ea-8ff0-55f74ac67de5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:41.776611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306dd38-7d82-45ea-8ff0-55f74ac67de5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:41.781345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:41.782131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:41.782421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:41.782759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] PATH_INFO: `/v3/users/65139c98d95b4b6493e6136bf74a619d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:41.783996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65139c98d95b4b6493e6136bf74a619d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:41.785951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:41.842059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65139c98d95b4b6493e6136bf74a619d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:41.842867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65139c98d95b4b6493e6136bf74a619d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:41.855647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65139c98d95b4b6493e6136bf74a619d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:41.862557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65139c98d95b4b6493e6136bf74a619d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:41.872994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65139c98d95b4b6493e6136bf74a619d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:41.879949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1037/2095] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:41 2022] DELETE /identity/v3/users/65139c98d95b4b6493e6136bf74a619d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:41.884850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cd71b-8fd9-47f0-981d-be603a083114 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:41.885432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cd71b-8fd9-47f0-981d-be603a083114 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:41.885432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cd71b-8fd9-47f0-981d-be603a083114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:41.905487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4e4cd71b-8fd9-47f0-981d-be603a083114 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:41.907011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4e4cd71b-8fd9-47f0-981d-be603a083114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:41.967834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1059/2096] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:41.978470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:41.980104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:42.055814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db76ca4-c290-423c-9f90-67831f75b143 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:42.064770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbddf3-00f2-40f8-a54e-dbfc8af88295 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:42.065420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbddf3-00f2-40f8-a54e-dbfc8af88295 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:42.065927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbddf3-00f2-40f8-a54e-dbfc8af88295 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:42.066326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbddf3-00f2-40f8-a54e-dbfc8af88295 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:42.072526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cd71b-8fd9-47f0-981d-be603a083114 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:42.074916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4e4cd71b-8fd9-47f0-981d-be603a083114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:42.077973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbddf3-00f2-40f8-a54e-dbfc8af88295 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:42.079423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbddf3-00f2-40f8-a54e-dbfc8af88295 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:42.089105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cd71b-8fd9-47f0-981d-be603a083114 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:42.090450 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1038/2097] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:42.094944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:42.095771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:42.096052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:42.096327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] PATH_INFO: `/v3/users/c16317f80360482a8021dc2a486c5c43` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:42.097541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c16317f80360482a8021dc2a486c5c43)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:42.099519 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:42.171920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c16317f80360482a8021dc2a486c5c43'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:42.172826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c16317f80360482a8021dc2a486c5c43'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:42.193598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c16317f80360482a8021dc2a486c5c43'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:42.204290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c16317f80360482a8021dc2a486c5c43'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:42.214765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c16317f80360482a8021dc2a486c5c43'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:42.227906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1060/2098] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:42 2022] DELETE /identity/v3/users/c16317f80360482a8021dc2a486c5c43 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:42.233433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438ca1c-b2c5-4d2e-8015-5831bb05295c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:42.233875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438ca1c-b2c5-4d2e-8015-5831bb05295c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:42.234466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438ca1c-b2c5-4d2e-8015-5831bb05295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:42.254252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:42.278003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e438ca1c-b2c5-4d2e-8015-5831bb05295c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:42.279857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e438ca1c-b2c5-4d2e-8015-5831bb05295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:42.356840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1039/2099] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:42.376186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:42.378457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:42.385021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:42.477352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438ca1c-b2c5-4d2e-8015-5831bb05295c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:42.478129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898e6e5-1dc2-4e97-957d-9ec0b7c6bce7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:42.479257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e438ca1c-b2c5-4d2e-8015-5831bb05295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:42.493123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90361906-7e41-48bf-b30f-a542079d6859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:42.493905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90361906-7e41-48bf-b30f-a542079d6859 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:42.494294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90361906-7e41-48bf-b30f-a542079d6859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:42.494819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90361906-7e41-48bf-b30f-a542079d6859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:42.505993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90361906-7e41-48bf-b30f-a542079d6859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:42.507668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90361906-7e41-48bf-b30f-a542079d6859 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:42.519144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438ca1c-b2c5-4d2e-8015-5831bb05295c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:42.526038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:42.527102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:42.527527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:42.527973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] PATH_INFO: `/v3/projects/72b2929a4d1749539a930c8847362f68` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:42.530648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72b2929a4d1749539a930c8847362f68)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:42.532878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1061/2100] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:42.533493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:42.601563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72b2929a4d1749539a930c8847362f68', 'request_id': 'req-4454174b-7129-4c8a-8cbf-58d9f104b4f1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:42.610228 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72b2929a4d1749539a930c8847362f68', 'request_id': 'req-4454174b-7129-4c8a-8cbf-58d9f104b4f1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:42.619572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72b2929a4d1749539a930c8847362f68', 'request_id': 'req-4454174b-7129-4c8a-8cbf-58d9f104b4f1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:42.633578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] The token cache is being invalidate because project 72b2929a4d1749539a930c8847362f68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:42.633942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:42.634592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-4454174b-7129-4c8a-8cbf-58d9f104b4f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:42.636637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1040/2101] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:42 2022] DELETE /identity/v3/projects/72b2929a4d1749539a930c8847362f68 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:42.642586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017b3ec-c6ae-4694-9e48-6cec7d758b49 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:42.642883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017b3ec-c6ae-4694-9e48-6cec7d758b49 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:42.643171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017b3ec-c6ae-4694-9e48-6cec7d758b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:42.664682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d017b3ec-c6ae-4694-9e48-6cec7d758b49 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:42.665942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d017b3ec-c6ae-4694-9e48-6cec7d758b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:42.731865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1062/2102] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:42.741323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c5772-c251-421e-8e61-1f9b89ecfde2 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:42.741887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c5772-c251-421e-8e61-1f9b89ecfde2 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:42.742229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c5772-c251-421e-8e61-1f9b89ecfde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:42.766107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a39c5772-c251-421e-8e61-1f9b89ecfde2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:42.767383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a39c5772-c251-421e-8e61-1f9b89ecfde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:42.777794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1041/2103] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:42.782153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017b3ec-c6ae-4694-9e48-6cec7d758b49 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:42.800315 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017b3ec-c6ae-4694-9e48-6cec7d758b49 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:42.806314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923912d6-0b43-4c58-ae4b-9d3c130a59bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:42.807497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923912d6-0b43-4c58-ae4b-9d3c130a59bc admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:42.807786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923912d6-0b43-4c58-ae4b-9d3c130a59bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:42.809206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923912d6-0b43-4c58-ae4b-9d3c130a59bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:42.811862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923912d6-0b43-4c58-ae4b-9d3c130a59bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:42.814050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923912d6-0b43-4c58-ae4b-9d3c130a59bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:42.816525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923912d6-0b43-4c58-ae4b-9d3c130a59bc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:42.846081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c5772-c251-421e-8e61-1f9b89ecfde2 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:42.846081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a39c5772-c251-421e-8e61-1f9b89ecfde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:42.847236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1063/2104] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:42.848943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923912d6-0b43-4c58-ae4b-9d3c130a59bc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:42.868907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c5772-c251-421e-8e61-1f9b89ecfde2 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:42.875637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923912d6-0b43-4c58-ae4b-9d3c130a59bc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:42.882190 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:42.883200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:42.883626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:42.884157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] PATH_INFO: `/v3/projects/8b172d22875140fc89b8b4fc749e218d` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:42.885574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b172d22875140fc89b8b4fc749e218d)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:42.885782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ca7a3-f4da-4d84-a162-ca73ba0bf675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:42.886855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ca7a3-f4da-4d84-a162-ca73ba0bf675 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:42.887233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ca7a3-f4da-4d84-a162-ca73ba0bf675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:42.887601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ca7a3-f4da-4d84-a162-ca73ba0bf675 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:42.888384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ca7a3-f4da-4d84-a162-ca73ba0bf675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:42.891298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ca7a3-f4da-4d84-a162-ca73ba0bf675 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:42.899951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:42.950404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1064/2105] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:42 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:42.955785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ca7a3-f4da-4d84-a162-ca73ba0bf675 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:42.996372 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b172d22875140fc89b8b4fc749e218d', 'request_id': 'req-5f3ce901-372f-41e3-ad78-d384e012ae27'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.010556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b172d22875140fc89b8b4fc749e218d', 'request_id': 'req-5f3ce901-372f-41e3-ad78-d384e012ae27'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.030270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b172d22875140fc89b8b4fc749e218d', 'request_id': 'req-5f3ce901-372f-41e3-ad78-d384e012ae27'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.047195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] The token cache is being invalidate because project 8b172d22875140fc89b8b4fc749e218d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:43.047488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.048311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.050642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1042/2106] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:42 2022] DELETE /identity/v3/projects/8b172d22875140fc89b8b4fc749e218d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:43.054522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:43.056353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:43.061530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ca7a3-f4da-4d84-a162-ca73ba0bf675 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:43.069207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:43.093251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:43.166961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefb6af-d2de-4d53-b822-c1edd94b5894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:43.167882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefb6af-d2de-4d53-b822-c1edd94b5894 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:43.167882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefb6af-d2de-4d53-b822-c1edd94b5894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:43.168820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeefb6af-d2de-4d53-b822-c1edd94b5894 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:43.169016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefb6af-d2de-4d53-b822-c1edd94b5894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:43.171086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeefb6af-d2de-4d53-b822-c1edd94b5894 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:43.171573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ce901-372f-41e3-ad78-d384e012ae27 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:43.177982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:43.179580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:43.180496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:43.180740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] PATH_INFO: `/v3/users/474a1b0893ee4b52a6fd1ff21b8b0d88` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:43.182625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=474a1b0893ee4b52a6fd1ff21b8b0d88)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:43.184900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:43.198294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1065/2107] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:42 2022] POST /identity/v3/users => generated 287 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:43.199954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefb6af-d2de-4d53-b822-c1edd94b5894 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:43.202520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aeefb6af-d2de-4d53-b822-c1edd94b5894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:43.215822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeefb6af-d2de-4d53-b822-c1edd94b5894 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:43.222492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d1c5-e6cf-4dbb-81f5-74a81a5a8fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:43.223273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d1c5-e6cf-4dbb-81f5-74a81a5a8fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:43.223687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d1c5-e6cf-4dbb-81f5-74a81a5a8fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:43.224029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75d1c5-e6cf-4dbb-81f5-74a81a5a8fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:43.224887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d1c5-e6cf-4dbb-81f5-74a81a5a8fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:43.226773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75d1c5-e6cf-4dbb-81f5-74a81a5a8fd6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:43.241657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1066/2108] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:43.242827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d1c5-e6cf-4dbb-81f5-74a81a5a8fd6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:43.247674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '474a1b0893ee4b52a6fd1ff21b8b0d88'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.248471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '474a1b0893ee4b52a6fd1ff21b8b0d88'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.253427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75d1c5-e6cf-4dbb-81f5-74a81a5a8fd6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:43.259303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b6789-8c4a-4b9b-9749-7bc005452843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:43.259935 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b6789-8c4a-4b9b-9749-7bc005452843 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:43.260223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b6789-8c4a-4b9b-9749-7bc005452843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:43.260503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b6789-8c4a-4b9b-9749-7bc005452843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:43.261222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b6789-8c4a-4b9b-9749-7bc005452843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:43.262888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b6789-8c4a-4b9b-9749-7bc005452843 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:43.266720 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '474a1b0893ee4b52a6fd1ff21b8b0d88'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.277436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1067/2109] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:43.277625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '474a1b0893ee4b52a6fd1ff21b8b0d88'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.279270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b6789-8c4a-4b9b-9749-7bc005452843 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:43.284216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-8a80b82a-cef9-41dd-9b85-de3aefd25698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '474a1b0893ee4b52a6fd1ff21b8b0d88'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.291346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1043/2110] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:43 2022] DELETE /identity/v3/users/474a1b0893ee4b52a6fd1ff21b8b0d88 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:43.292226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b6789-8c4a-4b9b-9749-7bc005452843 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:43.365287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3555dc-5b8f-435f-8a27-34c352392ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:43.366085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3555dc-5b8f-435f-8a27-34c352392ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:43.366543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3555dc-5b8f-435f-8a27-34c352392ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:43.366745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3555dc-5b8f-435f-8a27-34c352392ffb admin admin] PATH_INFO: `/v3/projects/0c385c0c5e784836adffdc0c75ee2558/users/6b276e46e849470292b663ebcc618588/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:43.386226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3555dc-5b8f-435f-8a27-34c352392ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c385c0c5e784836adffdc0c75ee2558, user_id=6b276e46e849470292b663ebcc618588, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:43.389162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3555dc-5b8f-435f-8a27-34c352392ffb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:43.404159 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1068/2111] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:43 2022] PUT /identity/v3/projects/0c385c0c5e784836adffdc0c75ee2558/users/6b276e46e849470292b663ebcc618588/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:43.416496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2010fc-4ec7-45cc-8017-3fdc02c3b775 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:43.417492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2010fc-4ec7-45cc-8017-3fdc02c3b775 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:43.418035 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2010fc-4ec7-45cc-8017-3fdc02c3b775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:43.474708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8f2010fc-4ec7-45cc-8017-3fdc02c3b775 None None] MFA Rules not processed for user `6b276e46e849470292b663ebcc618588`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:43.476184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f2010fc-4ec7-45cc-8017-3fdc02c3b775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:43.579409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1044/2112] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:43.590106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3555dc-5b8f-435f-8a27-34c352392ffb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:43.592926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe3555dc-5b8f-435f-8a27-34c352392ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:43.684354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2010fc-4ec7-45cc-8017-3fdc02c3b775 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:43.686086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f2010fc-4ec7-45cc-8017-3fdc02c3b775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:43.706154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3555dc-5b8f-435f-8a27-34c352392ffb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:43.711258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96c7bd-0025-42a6-97c5-18353f80d20b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:43.712382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96c7bd-0025-42a6-97c5-18353f80d20b service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:43.712957 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96c7bd-0025-42a6-97c5-18353f80d20b service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:43.713512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96c7bd-0025-42a6-97c5-18353f80d20b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:43.717431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:43.726655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96c7bd-0025-42a6-97c5-18353f80d20b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:43.729565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df96c7bd-0025-42a6-97c5-18353f80d20b service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:43.741582 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1069/2113] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:43.773813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96c7bd-0025-42a6-97c5-18353f80d20b service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:43.775036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-df96c7bd-0025-42a6-97c5-18353f80d20b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:43.793220 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:43.823125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2010fc-4ec7-45cc-8017-3fdc02c3b775 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:43.832519 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:43.833395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:43.833852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:43.834236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] PATH_INFO: `/v3/projects/68d993f9663645daa3fabc12ad9065c2` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:43.836106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68d993f9663645daa3fabc12ad9065c2)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:43.838911 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:43.882922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96c7bd-0025-42a6-97c5-18353f80d20b service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:43.889729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:43.890592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:43.890968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:43.891520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] PATH_INFO: `/v3/users/6b276e46e849470292b663ebcc618588` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:43.893101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b276e46e849470292b663ebcc618588)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:43.895979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:43.903751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68d993f9663645daa3fabc12ad9065c2', 'request_id': 'req-815ec511-e260-4954-9022-9b476a339f02'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.911198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68d993f9663645daa3fabc12ad9065c2', 'request_id': 'req-815ec511-e260-4954-9022-9b476a339f02'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.922326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68d993f9663645daa3fabc12ad9065c2', 'request_id': 'req-815ec511-e260-4954-9022-9b476a339f02'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.939304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] The token cache is being invalidate because project 68d993f9663645daa3fabc12ad9065c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:43.939717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.940495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.942527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1045/2114] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:43 2022] DELETE /identity/v3/projects/68d993f9663645daa3fabc12ad9065c2 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:43.948406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:43.951698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:43.970788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b276e46e849470292b663ebcc618588'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.973807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b276e46e849470292b663ebcc618588'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:43.990589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b276e46e849470292b663ebcc618588'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:44.001262 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b276e46e849470292b663ebcc618588'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:44.013875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f5c1d60e-f99a-4de6-afe7-c7f9292062cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b276e46e849470292b663ebcc618588'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:44.027484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1070/2115] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:43 2022] DELETE /identity/v3/users/6b276e46e849470292b663ebcc618588 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:44.029776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7d848-80b3-4174-9c1d-8d7d03161cbb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:44.029776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7d848-80b3-4174-9c1d-8d7d03161cbb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:44.029776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7d848-80b3-4174-9c1d-8d7d03161cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:44.048848 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815ec511-e260-4954-9022-9b476a339f02 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:44.052438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-84b7d848-80b3-4174-9c1d-8d7d03161cbb None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:44.053643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-84b7d848-80b3-4174-9c1d-8d7d03161cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:44.114558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:44.137334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1071/2116] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:44.142843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:44.143553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:44.143899 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:44.144239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] PATH_INFO: `/v3/users/801b79a2cb764879a5dd3fa01df074de` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:44.146082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=801b79a2cb764879a5dd3fa01df074de)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:44.147895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:44.150175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7d848-80b3-4174-9c1d-8d7d03161cbb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:44.151780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-84b7d848-80b3-4174-9c1d-8d7d03161cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:44.206877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:44.242057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '801b79a2cb764879a5dd3fa01df074de'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:44.243769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '801b79a2cb764879a5dd3fa01df074de'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:44.248265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7d848-80b3-4174-9c1d-8d7d03161cbb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:44.255641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eb31c-cb70-48d8-94eb-8087184a373f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:44.256634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eb31c-cb70-48d8-94eb-8087184a373f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:44.257026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eb31c-cb70-48d8-94eb-8087184a373f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:44.257398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9eb31c-cb70-48d8-94eb-8087184a373f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:44.260883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '801b79a2cb764879a5dd3fa01df074de'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:44.277280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9eb31c-cb70-48d8-94eb-8087184a373f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:44.280217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '801b79a2cb764879a5dd3fa01df074de'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:44.280355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9eb31c-cb70-48d8-94eb-8087184a373f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:44.290899 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '801b79a2cb764879a5dd3fa01df074de'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:44.294266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1072/2117] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:44.298656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1046/2118] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:43 2022] DELETE /identity/v3/users/801b79a2cb764879a5dd3fa01df074de => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:44.999082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eb31c-cb70-48d8-94eb-8087184a373f service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:45.000603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe9eb31c-cb70-48d8-94eb-8087184a373f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:45.017541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9eb31c-cb70-48d8-94eb-8087184a373f service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:45.097977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:45.100095 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:45.142617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:45.143497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:45.143878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:45.144253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] PATH_INFO: `/v3/projects/6691b682a75e42f69b8249e319aaa932` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:45.145919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6691b682a75e42f69b8249e319aaa932)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:45.148216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:45.226756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6691b682a75e42f69b8249e319aaa932', 'request_id': 'req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:45.228977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bdfec-e1d4-45c3-a904-a9f5f3a2fa95 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:45.236752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6691b682a75e42f69b8249e319aaa932', 'request_id': 'req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:45.244998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6691b682a75e42f69b8249e319aaa932', 'request_id': 'req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:45.265039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] The token cache is being invalidate because project 6691b682a75e42f69b8249e319aaa932 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:45.265337 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:45.266056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-66d4d3a9-29f5-4293-80bc-9d20f995c8a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:45.267827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1073/2119] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:44 2022] DELETE /identity/v3/projects/6691b682a75e42f69b8249e319aaa932 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:45.274356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6478-56e1-481e-a695-2b11a2084770 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:45.274640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6478-56e1-481e-a695-2b11a2084770 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:45.274973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f6478-56e1-481e-a695-2b11a2084770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:45.296355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-153f6478-56e1-481e-a695-2b11a2084770 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:45.297630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-153f6478-56e1-481e-a695-2b11a2084770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:45.308993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:45.310151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:45.310433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:45.310710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] PATH_INFO: `/v3/projects/0c385c0c5e784836adffdc0c75ee2558` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:45.312657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c385c0c5e784836adffdc0c75ee2558)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:45.315101 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:45.344441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:45.384378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c385c0c5e784836adffdc0c75ee2558', 'request_id': 'req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:45.387376 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1074/2120] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:45.395105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c385c0c5e784836adffdc0c75ee2558', 'request_id': 'req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:45.404051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c385c0c5e784836adffdc0c75ee2558', 'request_id': 'req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:45.427539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] The token cache is being invalidate because project 0c385c0c5e784836adffdc0c75ee2558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:45.427539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:45.427539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:45.427539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1047/2121] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:45 2022] DELETE /identity/v3/projects/0c385c0c5e784836adffdc0c75ee2558 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:45.469098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9bde3-ebd1-462f-86ab-8e65d8264f92 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:45.469377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9bde3-ebd1-462f-86ab-8e65d8264f92 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:45.469652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9bde3-ebd1-462f-86ab-8e65d8264f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:45.479201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:45.494591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b4b9bde3-ebd1-462f-86ab-8e65d8264f92 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:45.496737 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b4b9bde3-ebd1-462f-86ab-8e65d8264f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:45.599887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1075/2122] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:45.605075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:45.626017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86576f27-a0e3-43fb-bdf4-8bfc1e433cb2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:45.635147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c040a8-8739-45a1-82f7-a14f3ff43806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:45.636438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c040a8-8739-45a1-82f7-a14f3ff43806 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:45.637651 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c040a8-8739-45a1-82f7-a14f3ff43806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:45.638219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c040a8-8739-45a1-82f7-a14f3ff43806 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:45.641468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c040a8-8739-45a1-82f7-a14f3ff43806 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:45.642722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c040a8-8739-45a1-82f7-a14f3ff43806 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:45.645008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c040a8-8739-45a1-82f7-a14f3ff43806 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:45.664542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1048/2123] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:45.675811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9bde3-ebd1-462f-86ab-8e65d8264f92 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:45.688912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9bde3-ebd1-462f-86ab-8e65d8264f92 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:45.693869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7641bd5-f6e3-443d-a3a7-e9b513b12afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:45.694508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7641bd5-f6e3-443d-a3a7-e9b513b12afe admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:45.694926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7641bd5-f6e3-443d-a3a7-e9b513b12afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:45.695226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7641bd5-f6e3-443d-a3a7-e9b513b12afe admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:45.695899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7641bd5-f6e3-443d-a3a7-e9b513b12afe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:45.697880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7641bd5-f6e3-443d-a3a7-e9b513b12afe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:45.734928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1076/2124] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:45 2022] POST /identity/v3/projects => generated 367 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:45.759785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c040a8-8739-45a1-82f7-a14f3ff43806 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:45.805690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c040a8-8739-45a1-82f7-a14f3ff43806 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:45.943284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4853d34-3828-40ed-83f0-ff4c68b41c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:45.944606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4853d34-3828-40ed-83f0-ff4c68b41c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:45.947725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4853d34-3828-40ed-83f0-ff4c68b41c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:45.948677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4853d34-3828-40ed-83f0-ff4c68b41c23 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:45.951070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4853d34-3828-40ed-83f0-ff4c68b41c23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:45.961218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4853d34-3828-40ed-83f0-ff4c68b41c23 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:46.015762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1049/2125] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:45 2022] POST /identity/v3/users => generated 284 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:46.019429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7641bd5-f6e3-443d-a3a7-e9b513b12afe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:46.053819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7641bd5-f6e3-443d-a3a7-e9b513b12afe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:46.072776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e4d15-66c9-4301-99b3-205e316ec291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:46.073430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e4d15-66c9-4301-99b3-205e316ec291 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:46.073715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e4d15-66c9-4301-99b3-205e316ec291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:46.074238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e4d15-66c9-4301-99b3-205e316ec291 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:46.075247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e4d15-66c9-4301-99b3-205e316ec291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:46.077865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e4d15-66c9-4301-99b3-205e316ec291 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:46.098629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1077/2126] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:46.102038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4853d34-3828-40ed-83f0-ff4c68b41c23 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:46.118807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4853d34-3828-40ed-83f0-ff4c68b41c23 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:46.126638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f882b02d-95a6-4fb0-bbf2-c38a30663247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:46.127694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882b02d-95a6-4fb0-bbf2-c38a30663247 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:46.128150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882b02d-95a6-4fb0-bbf2-c38a30663247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:46.128621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f882b02d-95a6-4fb0-bbf2-c38a30663247 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:46.129972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882b02d-95a6-4fb0-bbf2-c38a30663247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:46.131761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f882b02d-95a6-4fb0-bbf2-c38a30663247 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:46.145767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1050/2127] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:46.150060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e4d15-66c9-4301-99b3-205e316ec291 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:46.169094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e4d15-66c9-4301-99b3-205e316ec291 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:46.175364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003d0a4-4cbe-462b-904c-a90245c544fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:46.176366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003d0a4-4cbe-462b-904c-a90245c544fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:46.176876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003d0a4-4cbe-462b-904c-a90245c544fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:46.177250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c003d0a4-4cbe-462b-904c-a90245c544fe admin admin] PATH_INFO: `/v3/projects/008f2ea45e5e4ef3995ad89621819ad7/users/8896a709512b43ed8963d78cee158878/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:46.196053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003d0a4-4cbe-462b-904c-a90245c544fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=008f2ea45e5e4ef3995ad89621819ad7, user_id=8896a709512b43ed8963d78cee158878, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:46.199782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c003d0a4-4cbe-462b-904c-a90245c544fe admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:46.237873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1078/2128] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:46 2022] PUT /identity/v3/projects/008f2ea45e5e4ef3995ad89621819ad7/users/8896a709512b43ed8963d78cee158878/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:46.248024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574eb34-a8b6-4bb7-8236-9ebb1c52f1bc None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:46.248340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574eb34-a8b6-4bb7-8236-9ebb1c52f1bc None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:46.248578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7574eb34-a8b6-4bb7-8236-9ebb1c52f1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:46.279813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7574eb34-a8b6-4bb7-8236-9ebb1c52f1bc None None] MFA Rules not processed for user `8896a709512b43ed8963d78cee158878`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:46.280811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7574eb34-a8b6-4bb7-8236-9ebb1c52f1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:46.382261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1051/2129] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:46 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:46.394594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003d0a4-4cbe-462b-904c-a90245c544fe admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:46.396812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c003d0a4-4cbe-462b-904c-a90245c544fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:46.515023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c003d0a4-4cbe-462b-904c-a90245c544fe admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:46.519625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a7faf3-4eca-4962-985b-f02ec0b447c7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c6943686c843e588df14f7c6dffb8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:46.520317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7faf3-4eca-4962-985b-f02ec0b447c7 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:46.520623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7faf3-4eca-4962-985b-f02ec0b447c7 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:46.520928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a7faf3-4eca-4962-985b-f02ec0b447c7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:46.545913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7faf3-4eca-4962-985b-f02ec0b447c7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:46.546620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a7faf3-4eca-4962-985b-f02ec0b447c7 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:46.557686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1079/2130] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:46 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:46.631703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e81699-908e-4bb4-b63e-9921637756b7 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:46.632298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e81699-908e-4bb4-b63e-9921637756b7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:46.632639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e81699-908e-4bb4-b63e-9921637756b7 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:46.634273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1052/2131] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:26:46.640488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a7db0-bd67-4481-bc8b-e43addead437 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:46.641062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a7db0-bd67-4481-bc8b-e43addead437 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:46.641331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a7db0-bd67-4481-bc8b-e43addead437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:46.667145 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e28a7db0-bd67-4481-bc8b-e43addead437 None None] MFA Rules not processed for user `51a5093a60ef45919ab6e159adf58336`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:46.668677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e28a7db0-bd67-4481-bc8b-e43addead437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:46.692082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e81699-908e-4bb4-b63e-9921637756b7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:46.699349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-35e81699-908e-4bb4-b63e-9921637756b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:46.701699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:46.707115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:46.764368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1080/2132] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:26:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:46.777036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a7db0-bd67-4481-bc8b-e43addead437 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:46.778605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e28a7db0-bd67-4481-bc8b-e43addead437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:46.793633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a7db0-bd67-4481-bc8b-e43addead437 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:46.800773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ffca4-a58e-46a1-bba7-2ee8e5cea5cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:46.801472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ffca4-a58e-46a1-bba7-2ee8e5cea5cb service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:46.801680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e81699-908e-4bb4-b63e-9921637756b7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:46.801827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ffca4-a58e-46a1-bba7-2ee8e5cea5cb service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:46.802089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ffca4-a58e-46a1-bba7-2ee8e5cea5cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:46.808553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:46.809416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:46.809679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:46.810115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] PATH_INFO: `/v3/users/8fdc64e58cb041f98d9ea4b8c8cd9e60` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:46.812659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fdc64e58cb041f98d9ea4b8c8cd9e60)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:46.816117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:46.830157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ffca4-a58e-46a1-bba7-2ee8e5cea5cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:46.831945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ffca4-a58e-46a1-bba7-2ee8e5cea5cb service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:46.847209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1081/2133] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:26:46.923197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fdc64e58cb041f98d9ea4b8c8cd9e60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:46.924875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fdc64e58cb041f98d9ea4b8c8cd9e60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:46.943158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fdc64e58cb041f98d9ea4b8c8cd9e60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:46.953925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fdc64e58cb041f98d9ea4b8c8cd9e60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:46.966076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fa77f851-3312-4013-a739-f47dade5e2d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fdc64e58cb041f98d9ea4b8c8cd9e60'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:46.975942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1053/2134] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:46 2022] DELETE /identity/v3/users/8fdc64e58cb041f98d9ea4b8c8cd9e60 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:47.250841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ffca4-a58e-46a1-bba7-2ee8e5cea5cb service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:47.280379 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ffca4-a58e-46a1-bba7-2ee8e5cea5cb service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:47.364583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:47.365249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:47.365539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:47.365866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] PATH_INFO: `/v3/projects/500111b9c9ca4d8893e64c488668a491` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:47.367250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] RBAC: Authorizing `identity:delete_project(project_id=500111b9c9ca4d8893e64c488668a491)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:47.369185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:47.417257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '500111b9c9ca4d8893e64c488668a491', 'request_id': 'req-e41bb80e-ebde-4509-8616-4a65ea79451f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:47.423916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '500111b9c9ca4d8893e64c488668a491', 'request_id': 'req-e41bb80e-ebde-4509-8616-4a65ea79451f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:47.429570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '500111b9c9ca4d8893e64c488668a491', 'request_id': 'req-e41bb80e-ebde-4509-8616-4a65ea79451f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:47.443522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] The token cache is being invalidate because project 500111b9c9ca4d8893e64c488668a491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:47.443837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:47.444448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e41bb80e-ebde-4509-8616-4a65ea79451f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:47.446047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1082/2135] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:47 2022] DELETE /identity/v3/projects/500111b9c9ca4d8893e64c488668a491 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:47.451857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e394-795b-439e-9aef-33ec03398628 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:47.452430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e394-795b-439e-9aef-33ec03398628 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:47.452586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5e394-795b-439e-9aef-33ec03398628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:47.476223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-20e5e394-795b-439e-9aef-33ec03398628 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:47.477496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20e5e394-795b-439e-9aef-33ec03398628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:47.553935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1054/2136] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:47.555709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79d13b-ae0c-4e46-97eb-0335150a4214 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:47.555992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79d13b-ae0c-4e46-97eb-0335150a4214 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:47.556267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79d13b-ae0c-4e46-97eb-0335150a4214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:47.579197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3d79d13b-ae0c-4e46-97eb-0335150a4214 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:47.580553 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d79d13b-ae0c-4e46-97eb-0335150a4214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:47.594703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1083/2137] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:47.596354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e394-795b-439e-9aef-33ec03398628 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:47.607041 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5e394-795b-439e-9aef-33ec03398628 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:47.612865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131773e8-896c-4302-9770-70625c8534aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:47.613641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131773e8-896c-4302-9770-70625c8534aa admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:47.613966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131773e8-896c-4302-9770-70625c8534aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:47.614244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131773e8-896c-4302-9770-70625c8534aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:47.614952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131773e8-896c-4302-9770-70625c8534aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:47.615347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131773e8-896c-4302-9770-70625c8534aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:47.617021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131773e8-896c-4302-9770-70625c8534aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:47.631185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1055/2138] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:47.635758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79d13b-ae0c-4e46-97eb-0335150a4214 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:47.645285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79d13b-ae0c-4e46-97eb-0335150a4214 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:47.650615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536139d3-ce67-46ba-b518-0f32df536dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:47.651257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536139d3-ce67-46ba-b518-0f32df536dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:47.651534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536139d3-ce67-46ba-b518-0f32df536dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:47.651809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536139d3-ce67-46ba-b518-0f32df536dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:47.652487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536139d3-ce67-46ba-b518-0f32df536dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:47.654438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536139d3-ce67-46ba-b518-0f32df536dc3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:47.688927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1084/2139] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:47 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:47.690079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131773e8-896c-4302-9770-70625c8534aa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:47.702015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131773e8-896c-4302-9770-70625c8534aa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:47.710572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:47.793538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce5d46-264c-42c4-a5ee-81cc603820f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:47.794258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce5d46-264c-42c4-a5ee-81cc603820f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:47.794595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce5d46-264c-42c4-a5ee-81cc603820f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:47.794988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ce5d46-264c-42c4-a5ee-81cc603820f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:47.795758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce5d46-264c-42c4-a5ee-81cc603820f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:47.797986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ce5d46-264c-42c4-a5ee-81cc603820f2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:47.825385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1056/2140] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:47 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:47.829907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536139d3-ce67-46ba-b518-0f32df536dc3 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:47.841257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536139d3-ce67-46ba-b518-0f32df536dc3 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:47.846765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5b172-bb55-44cd-9fe2-ec8c1df45006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:47.847466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5b172-bb55-44cd-9fe2-ec8c1df45006 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:47.847728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5b172-bb55-44cd-9fe2-ec8c1df45006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:47.848001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5b172-bb55-44cd-9fe2-ec8c1df45006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:47.848687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5b172-bb55-44cd-9fe2-ec8c1df45006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:47.850361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5b172-bb55-44cd-9fe2-ec8c1df45006 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:47.861610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1085/2141] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:47.865975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce5d46-264c-42c4-a5ee-81cc603820f2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:47.877011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ce5d46-264c-42c4-a5ee-81cc603820f2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:47.882737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f60f5e-6601-4fc3-999e-dce071b40fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:47.883697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f60f5e-6601-4fc3-999e-dce071b40fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:47.884040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f60f5e-6601-4fc3-999e-dce071b40fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:47.884372 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f60f5e-6601-4fc3-999e-dce071b40fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:47.885629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f60f5e-6601-4fc3-999e-dce071b40fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:47.887632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f60f5e-6601-4fc3-999e-dce071b40fc3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:47.908934 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1057/2142] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:47.910285 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5b172-bb55-44cd-9fe2-ec8c1df45006 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:47.916926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:47.921454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5b172-bb55-44cd-9fe2-ec8c1df45006 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:47.928148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74c9b0-ced3-4e38-9716-e15aafb20472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:47.928821 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74c9b0-ced3-4e38-9716-e15aafb20472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:47.929481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74c9b0-ced3-4e38-9716-e15aafb20472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:47.929967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74c9b0-ced3-4e38-9716-e15aafb20472 admin admin] PATH_INFO: `/v3/projects/483f46736695483aadd19d64aa313258/users/9e7ec1679c2a45829a7be32cb85964e2/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:47.946475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74c9b0-ced3-4e38-9716-e15aafb20472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=483f46736695483aadd19d64aa313258, user_id=9e7ec1679c2a45829a7be32cb85964e2, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:47.949192 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a74c9b0-ced3-4e38-9716-e15aafb20472 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:47.964546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1086/2143] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:47 2022] PUT /identity/v3/projects/483f46736695483aadd19d64aa313258/users/9e7ec1679c2a45829a7be32cb85964e2/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:47.975973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53693-487c-4899-97ea-45c29a2bfdb9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:47.976245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53693-487c-4899-97ea-45c29a2bfdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:47.976550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e53693-487c-4899-97ea-45c29a2bfdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:48.007287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-95e53693-487c-4899-97ea-45c29a2bfdb9 None None] MFA Rules not processed for user `9e7ec1679c2a45829a7be32cb85964e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:48.008530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-95e53693-487c-4899-97ea-45c29a2bfdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:48.090294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1058/2144] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:48.095477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74c9b0-ced3-4e38-9716-e15aafb20472 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:48.200512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a74c9b0-ced3-4e38-9716-e15aafb20472 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:48.205910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d05d20-f97c-411f-8b31-e07357db652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:48.206613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d05d20-f97c-411f-8b31-e07357db652e admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:48.206891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d05d20-f97c-411f-8b31-e07357db652e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:48.207163 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d05d20-f97c-411f-8b31-e07357db652e admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:48.207838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d05d20-f97c-411f-8b31-e07357db652e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:48.209773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d05d20-f97c-411f-8b31-e07357db652e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:48.242017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1087/2145] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:48 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:48.246030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e53693-487c-4899-97ea-45c29a2bfdb9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:48.257503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e53693-487c-4899-97ea-45c29a2bfdb9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:48.320690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be4776-9a7b-4d59-9d93-4c7f148fc7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:48.321327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4776-9a7b-4d59-9d93-4c7f148fc7f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:48.321608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4776-9a7b-4d59-9d93-4c7f148fc7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:48.321933 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be4776-9a7b-4d59-9d93-4c7f148fc7f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:48.322622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be4776-9a7b-4d59-9d93-4c7f148fc7f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:48.324545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be4776-9a7b-4d59-9d93-4c7f148fc7f2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:48.363157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1059/2146] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:48 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:48.364461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d05d20-f97c-411f-8b31-e07357db652e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:48.380138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d05d20-f97c-411f-8b31-e07357db652e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:48.400736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f163f2-74b8-434a-9a7f-c5738ab4859e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:48.401463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f163f2-74b8-434a-9a7f-c5738ab4859e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:48.401764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f163f2-74b8-434a-9a7f-c5738ab4859e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:48.402067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f163f2-74b8-434a-9a7f-c5738ab4859e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:48.402806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f163f2-74b8-434a-9a7f-c5738ab4859e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:48.404834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f163f2-74b8-434a-9a7f-c5738ab4859e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:48.417168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1088/2147] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:48.421318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be4776-9a7b-4d59-9d93-4c7f148fc7f2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:48.437222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be4776-9a7b-4d59-9d93-4c7f148fc7f2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:48.447772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82384537-25e9-47c2-8ce6-d7c729d06eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:48.448599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82384537-25e9-47c2-8ce6-d7c729d06eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:48.448887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82384537-25e9-47c2-8ce6-d7c729d06eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:48.449167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82384537-25e9-47c2-8ce6-d7c729d06eae admin admin] PATH_INFO: `/v3/projects/45eadfae928540368d7ad81f3090e961/users/da21f8ba830e4ee39d7759123ae3e382/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:48.452996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:48.470812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82384537-25e9-47c2-8ce6-d7c729d06eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=45eadfae928540368d7ad81f3090e961, user_id=da21f8ba830e4ee39d7759123ae3e382, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:48.473788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82384537-25e9-47c2-8ce6-d7c729d06eae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:48.489246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1060/2148] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:48 2022] PUT /identity/v3/projects/45eadfae928540368d7ad81f3090e961/users/da21f8ba830e4ee39d7759123ae3e382/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:48.493328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f163f2-74b8-434a-9a7f-c5738ab4859e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:48.505820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f163f2-74b8-434a-9a7f-c5738ab4859e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:48.582243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dec470-c41c-4c09-b12a-6b02a23ac284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:48.583288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dec470-c41c-4c09-b12a-6b02a23ac284 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:48.583732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dec470-c41c-4c09-b12a-6b02a23ac284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:48.584173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dec470-c41c-4c09-b12a-6b02a23ac284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:48.585317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dec470-c41c-4c09-b12a-6b02a23ac284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:48.588218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dec470-c41c-4c09-b12a-6b02a23ac284 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:48.613771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1089/2149] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:48.618100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82384537-25e9-47c2-8ce6-d7c729d06eae admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:48.629047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82384537-25e9-47c2-8ce6-d7c729d06eae admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:48.634072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7761d0-a83a-4643-bbf3-0aee0a592e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:48.634829 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7761d0-a83a-4643-bbf3-0aee0a592e0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:48.635118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7761d0-a83a-4643-bbf3-0aee0a592e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:48.635396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7761d0-a83a-4643-bbf3-0aee0a592e0b admin admin] PATH_INFO: `/v3/projects/45eadfae928540368d7ad81f3090e961/users/da21f8ba830e4ee39d7759123ae3e382/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:48.637509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7761d0-a83a-4643-bbf3-0aee0a592e0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=45eadfae928540368d7ad81f3090e961, user_id=da21f8ba830e4ee39d7759123ae3e382, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:48.640320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7761d0-a83a-4643-bbf3-0aee0a592e0b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:48.653074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1061/2150] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:48 2022] PUT /identity/v3/projects/45eadfae928540368d7ad81f3090e961/users/da21f8ba830e4ee39d7759123ae3e382/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:48.665812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea71ca2-7dea-4140-800c-949d47812778 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:48.665812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea71ca2-7dea-4140-800c-949d47812778 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:48.665812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea71ca2-7dea-4140-800c-949d47812778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:48.707446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1ea71ca2-7dea-4140-800c-949d47812778 None None] MFA Rules not processed for user `da21f8ba830e4ee39d7759123ae3e382`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:48.708590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1ea71ca2-7dea-4140-800c-949d47812778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:48.755170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:48.786448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1090/2151] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:48.797994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7761d0-a83a-4643-bbf3-0aee0a592e0b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:48.799812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9d7761d0-a83a-4643-bbf3-0aee0a592e0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:48.888625 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7761d0-a83a-4643-bbf3-0aee0a592e0b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:48.893570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecee43a-f7d2-4d89-8805-b596539b5d54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:48.894269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee43a-f7d2-4d89-8805-b596539b5d54 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:48.894558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee43a-f7d2-4d89-8805-b596539b5d54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:48.894831 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecee43a-f7d2-4d89-8805-b596539b5d54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:48.906934 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecee43a-f7d2-4d89-8805-b596539b5d54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:48.908695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecee43a-f7d2-4d89-8805-b596539b5d54 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:48.921628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1062/2152] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:48.994743 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea71ca2-7dea-4140-800c-949d47812778 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:49.064493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea71ca2-7dea-4140-800c-949d47812778 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:49.069313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:49.069981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:49.070278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:49.070554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] PATH_INFO: `/v3/users/9e7ec1679c2a45829a7be32cb85964e2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:49.071974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e7ec1679c2a45829a7be32cb85964e2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:49.073928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:49.125338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e7ec1679c2a45829a7be32cb85964e2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.126027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e7ec1679c2a45829a7be32cb85964e2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.138099 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e7ec1679c2a45829a7be32cb85964e2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.146200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e7ec1679c2a45829a7be32cb85964e2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.156247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e7ec1679c2a45829a7be32cb85964e2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.163246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1091/2153] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:48 2022] DELETE /identity/v3/users/9e7ec1679c2a45829a7be32cb85964e2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:49.169775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216633d7-3614-4daf-9169-1a0870f27bef None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:49.170060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216633d7-3614-4daf-9169-1a0870f27bef None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:49.170364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216633d7-3614-4daf-9169-1a0870f27bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:49.189920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-216633d7-3614-4daf-9169-1a0870f27bef None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:49.191126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-216633d7-3614-4daf-9169-1a0870f27bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:49.258465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1063/2154] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:49.270193 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:49.272520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:49.335058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326167ec-4ae1-4afb-9df8-3222e91fba09 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:49.339223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eab988-0ad8-4f43-a48c-132fa1f701e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:49.339870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eab988-0ad8-4f43-a48c-132fa1f701e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:49.340155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eab988-0ad8-4f43-a48c-132fa1f701e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:49.340432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eab988-0ad8-4f43-a48c-132fa1f701e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:49.350875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eab988-0ad8-4f43-a48c-132fa1f701e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:49.352326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eab988-0ad8-4f43-a48c-132fa1f701e7 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:49.356585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:49.365167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1092/2155] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:49.689965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216633d7-3614-4daf-9169-1a0870f27bef None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:49.695863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-216633d7-3614-4daf-9169-1a0870f27bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:49.724163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216633d7-3614-4daf-9169-1a0870f27bef None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:49.729365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:49.730098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:49.730382 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:49.730828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] PATH_INFO: `/v3/projects/483f46736695483aadd19d64aa313258` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:49.732046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=483f46736695483aadd19d64aa313258)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:49.734688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:49.798591 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '483f46736695483aadd19d64aa313258', 'request_id': 'req-1a2aef34-7928-4d8e-b60b-089764c3a025'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.798924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eab988-0ad8-4f43-a48c-132fa1f701e7 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:49.800354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-57eab988-0ad8-4f43-a48c-132fa1f701e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:49.804566 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '483f46736695483aadd19d64aa313258', 'request_id': 'req-1a2aef34-7928-4d8e-b60b-089764c3a025'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.814018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eab988-0ad8-4f43-a48c-132fa1f701e7 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:49.816122 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '483f46736695483aadd19d64aa313258', 'request_id': 'req-1a2aef34-7928-4d8e-b60b-089764c3a025'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.824549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:49.830932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] The token cache is being invalidate because project 483f46736695483aadd19d64aa313258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:49.831423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.832781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.835012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1064/2156] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:49 2022] DELETE /identity/v3/projects/483f46736695483aadd19d64aa313258 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:49.840400 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:49.842424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:49.890767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:49.891434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:49.891716 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:49.891989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] PATH_INFO: `/v3/users/bc1a0ebb775a46eeb21e6cbf2a599bbf` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:49.893251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc1a0ebb775a46eeb21e6cbf2a599bbf)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:49.896279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:49.948887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2aef34-7928-4d8e-b60b-089764c3a025 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:49.954348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:49.955026 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:49.955305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:49.955582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] PATH_INFO: `/v3/users/da21f8ba830e4ee39d7759123ae3e382` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:49.956861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=da21f8ba830e4ee39d7759123ae3e382)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:49.958935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:49.963679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc1a0ebb775a46eeb21e6cbf2a599bbf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.964638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc1a0ebb775a46eeb21e6cbf2a599bbf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.986840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc1a0ebb775a46eeb21e6cbf2a599bbf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:49.997448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc1a0ebb775a46eeb21e6cbf2a599bbf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.008027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b1bed49-551e-42bc-81f8-5daedb86df1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc1a0ebb775a46eeb21e6cbf2a599bbf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.015010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1093/2157] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:49 2022] DELETE /identity/v3/users/bc1a0ebb775a46eeb21e6cbf2a599bbf => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:50.020270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcfc3-8b21-4d92-8e2b-3e5f275412e8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:50.020691 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcfc3-8b21-4d92-8e2b-3e5f275412e8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:50.021097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dcfc3-8b21-4d92-8e2b-3e5f275412e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:50.023637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da21f8ba830e4ee39d7759123ae3e382'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.024335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da21f8ba830e4ee39d7759123ae3e382'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.041089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da21f8ba830e4ee39d7759123ae3e382'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.043676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b61dcfc3-8b21-4d92-8e2b-3e5f275412e8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:50.045145 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b61dcfc3-8b21-4d92-8e2b-3e5f275412e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:50.050737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da21f8ba830e4ee39d7759123ae3e382'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.066219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da21f8ba830e4ee39d7759123ae3e382'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.077797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1065/2158] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:49 2022] DELETE /identity/v3/users/da21f8ba830e4ee39d7759123ae3e382 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:50.128018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1094/2159] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:50.138543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:50.140193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:50.216153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d04cf4-3b5e-4f5d-bab1-aa345da53fbb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:50.221445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86972e3-1aed-4cd3-a570-eaddc12ff970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:50.222054 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86972e3-1aed-4cd3-a570-eaddc12ff970 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:50.222271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86972e3-1aed-4cd3-a570-eaddc12ff970 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:50.222595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86972e3-1aed-4cd3-a570-eaddc12ff970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:50.266685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:50.305329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86972e3-1aed-4cd3-a570-eaddc12ff970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:50.306851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86972e3-1aed-4cd3-a570-eaddc12ff970 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:50.317932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1066/2160] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:50.404127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dcfc3-8b21-4d92-8e2b-3e5f275412e8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:50.407021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b61dcfc3-8b21-4d92-8e2b-3e5f275412e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:50.450445 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dcfc3-8b21-4d92-8e2b-3e5f275412e8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:50.456180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:50.458323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:50.459065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:50.459782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] PATH_INFO: `/v3/projects/45eadfae928540368d7ad81f3090e961` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:50.462948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=45eadfae928540368d7ad81f3090e961)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:50.468063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:50.496061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:50.584653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45eadfae928540368d7ad81f3090e961', 'request_id': 'req-0d28e702-ba88-4840-a744-2f542471eb9a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.596903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45eadfae928540368d7ad81f3090e961', 'request_id': 'req-0d28e702-ba88-4840-a744-2f542471eb9a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.605108 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45eadfae928540368d7ad81f3090e961', 'request_id': 'req-0d28e702-ba88-4840-a744-2f542471eb9a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.626631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] The token cache is being invalidate because project 45eadfae928540368d7ad81f3090e961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:50.626923 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.627594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0d28e702-ba88-4840-a744-2f542471eb9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.629379 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1095/2161] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:50 2022] DELETE /identity/v3/projects/45eadfae928540368d7ad81f3090e961 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:50.634393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af91b5c-da89-4252-813f-31bb42af3828 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:50.634770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af91b5c-da89-4252-813f-31bb42af3828 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:50.635131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af91b5c-da89-4252-813f-31bb42af3828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:50.661247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2af91b5c-da89-4252-813f-31bb42af3828 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:50.662944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2af91b5c-da89-4252-813f-31bb42af3828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:50.741170 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1067/2162] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:50.749493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c626b8-81de-4f3e-a206-a4f42e8f5fc3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:50.749803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c626b8-81de-4f3e-a206-a4f42e8f5fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:50.750085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c626b8-81de-4f3e-a206-a4f42e8f5fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:50.771723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-24c626b8-81de-4f3e-a206-a4f42e8f5fc3 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:50.773054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-24c626b8-81de-4f3e-a206-a4f42e8f5fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:50.787551 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af91b5c-da89-4252-813f-31bb42af3828 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:50.789319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2af91b5c-da89-4252-813f-31bb42af3828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:50.789878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1096/2163] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:50.794443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c626b8-81de-4f3e-a206-a4f42e8f5fc3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:50.806297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c626b8-81de-4f3e-a206-a4f42e8f5fc3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:50.807529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af91b5c-da89-4252-813f-31bb42af3828 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:50.813625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49ec65-6fe9-4ae7-8390-f27556434556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:50.814325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49ec65-6fe9-4ae7-8390-f27556434556 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:50.814700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49ec65-6fe9-4ae7-8390-f27556434556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:50.815071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49ec65-6fe9-4ae7-8390-f27556434556 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:50.815680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:50.816020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49ec65-6fe9-4ae7-8390-f27556434556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:50.816226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:50.816505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49ec65-6fe9-4ae7-8390-f27556434556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:50.816643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:50.816927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] PATH_INFO: `/v3/projects/432fa11cbca14210b3f2b31f029d4414` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:50.818467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49ec65-6fe9-4ae7-8390-f27556434556 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:50.819286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] RBAC: Authorizing `identity:delete_project(project_id=432fa11cbca14210b3f2b31f029d4414)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:50.822088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:50.839330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1097/2164] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:50.841042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49ec65-6fe9-4ae7-8390-f27556434556 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:50.857223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49ec65-6fe9-4ae7-8390-f27556434556 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:50.870449 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7700a91-feea-4cd9-9029-1035984ff653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:50.871567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7700a91-feea-4cd9-9029-1035984ff653 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:50.871856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7700a91-feea-4cd9-9029-1035984ff653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:50.872134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7700a91-feea-4cd9-9029-1035984ff653 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:50.872881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7700a91-feea-4cd9-9029-1035984ff653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:50.875065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7700a91-feea-4cd9-9029-1035984ff653 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:50.909943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432fa11cbca14210b3f2b31f029d4414', 'request_id': 'req-99ebd54f-a318-40e1-84db-13ba1c430432'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.914395 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1098/2165] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:50 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:50.915845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7700a91-feea-4cd9-9029-1035984ff653 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:50.915972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432fa11cbca14210b3f2b31f029d4414', 'request_id': 'req-99ebd54f-a318-40e1-84db-13ba1c430432'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.926268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432fa11cbca14210b3f2b31f029d4414', 'request_id': 'req-99ebd54f-a318-40e1-84db-13ba1c430432'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.942891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] The token cache is being invalidate because project 432fa11cbca14210b3f2b31f029d4414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:50.943185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.943926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-99ebd54f-a318-40e1-84db-13ba1c430432 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:50.945534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1068/2166] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:50 2022] DELETE /identity/v3/projects/432fa11cbca14210b3f2b31f029d4414 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:50.950660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3811d-bfc0-47b4-8eb5-005580cdb739 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:50.950957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3811d-bfc0-47b4-8eb5-005580cdb739 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:50.951211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3811d-bfc0-47b4-8eb5-005580cdb739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:50.980181 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bdf3811d-bfc0-47b4-8eb5-005580cdb739 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:50.981891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdf3811d-bfc0-47b4-8eb5-005580cdb739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:51.002674 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7700a91-feea-4cd9-9029-1035984ff653 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.008837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6364d2-7849-4f67-bbd2-18f3cd7aeae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.009487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6364d2-7849-4f67-bbd2-18f3cd7aeae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.009830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6364d2-7849-4f67-bbd2-18f3cd7aeae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.010069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6364d2-7849-4f67-bbd2-18f3cd7aeae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.010757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6364d2-7849-4f67-bbd2-18f3cd7aeae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.012826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6364d2-7849-4f67-bbd2-18f3cd7aeae9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.033726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:51.047769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1099/2167] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:50 2022] POST /identity/v3/users => generated 293 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:51.054424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6364d2-7849-4f67-bbd2-18f3cd7aeae9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.054424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ab6364d2-7849-4f67-bbd2-18f3cd7aeae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:51.058082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1069/2168] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:51.066429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac92ad5-4976-4534-b64a-2f6b89d868c4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.066721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac92ad5-4976-4534-b64a-2f6b89d868c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.066995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac92ad5-4976-4534-b64a-2f6b89d868c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.067871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6364d2-7849-4f67-bbd2-18f3cd7aeae9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.072610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d10935-a8e6-4389-a613-645b09930643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.073246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d10935-a8e6-4389-a613-645b09930643 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.073531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d10935-a8e6-4389-a613-645b09930643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.073845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d10935-a8e6-4389-a613-645b09930643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.074563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d10935-a8e6-4389-a613-645b09930643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.077632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d10935-a8e6-4389-a613-645b09930643 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.090912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bac92ad5-4976-4534-b64a-2f6b89d868c4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:51.092025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1100/2169] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:51.092171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bac92ad5-4976-4534-b64a-2f6b89d868c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:51.096662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d10935-a8e6-4389-a613-645b09930643 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.100752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1070/2170] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:51.104924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac92ad5-4976-4534-b64a-2f6b89d868c4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.106916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d10935-a8e6-4389-a613-645b09930643 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.112245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a70f5-3f92-412d-a2a4-f58fb081c29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.112931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a70f5-3f92-412d-a2a4-f58fb081c29d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.113210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a70f5-3f92-412d-a2a4-f58fb081c29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.113475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65a70f5-3f92-412d-a2a4-f58fb081c29d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.114227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a70f5-3f92-412d-a2a4-f58fb081c29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.114349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac92ad5-4976-4534-b64a-2f6b89d868c4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.115846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65a70f5-3f92-412d-a2a4-f58fb081c29d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.121120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2da3e-e762-4258-8b48-7a7565902d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.121984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2da3e-e762-4258-8b48-7a7565902d71 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.122123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2da3e-e762-4258-8b48-7a7565902d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.122333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c2da3e-e762-4258-8b48-7a7565902d71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.123082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2da3e-e762-4258-8b48-7a7565902d71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:51.123386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2da3e-e762-4258-8b48-7a7565902d71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.125060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c2da3e-e762-4258-8b48-7a7565902d71 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.127392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1101/2171] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:51.131555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a70f5-3f92-412d-a2a4-f58fb081c29d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.141428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65a70f5-3f92-412d-a2a4-f58fb081c29d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.142348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1071/2172] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:26:51.146752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2da3e-e762-4258-8b48-7a7565902d71 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.152267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdeef66-a851-4c38-8092-921ec116344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.153017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdeef66-a851-4c38-8092-921ec116344c admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.153281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdeef66-a851-4c38-8092-921ec116344c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.153552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdeef66-a851-4c38-8092-921ec116344c admin admin] PATH_INFO: `/v3/projects/3bf8703dfd3f4b0e97874b0520607d10/users/a6559e42d56843ffa9b1645237129ebd/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.157915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c2da3e-e762-4258-8b48-7a7565902d71 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.163415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de26e7-4384-4a42-9954-c95255541d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.164046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de26e7-4384-4a42-9954-c95255541d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.164330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de26e7-4384-4a42-9954-c95255541d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.164603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de26e7-4384-4a42-9954-c95255541d73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.165270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de26e7-4384-4a42-9954-c95255541d73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.167199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de26e7-4384-4a42-9954-c95255541d73 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.169651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdeef66-a851-4c38-8092-921ec116344c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf8703dfd3f4b0e97874b0520607d10, user_id=a6559e42d56843ffa9b1645237129ebd, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.173222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdeef66-a851-4c38-8092-921ec116344c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.190878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1102/2173] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:51 2022] PUT /identity/v3/projects/3bf8703dfd3f4b0e97874b0520607d10/users/a6559e42d56843ffa9b1645237129ebd/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:51.199751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ae158-4611-4ca7-9636-47f92b6d1f8c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.200114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ae158-4611-4ca7-9636-47f92b6d1f8c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.200514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ae158-4611-4ca7-9636-47f92b6d1f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.202319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1072/2174] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:51 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:51.206622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de26e7-4384-4a42-9954-c95255541d73 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.208334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:51.238290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1b8ae158-4611-4ca7-9636-47f92b6d1f8c None None] MFA Rules not processed for user `a6559e42d56843ffa9b1645237129ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:51.240280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b8ae158-4611-4ca7-9636-47f92b6d1f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:51.277162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de26e7-4384-4a42-9954-c95255541d73 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.282052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bd2eb-8108-4457-8005-165460438f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.282674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bd2eb-8108-4457-8005-165460438f46 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.283027 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bd2eb-8108-4457-8005-165460438f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.283281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bd2eb-8108-4457-8005-165460438f46 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.284017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bd2eb-8108-4457-8005-165460438f46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.285985 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bd2eb-8108-4457-8005-165460438f46 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.319770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1073/2175] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:51 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:51.321378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bd2eb-8108-4457-8005-165460438f46 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.335792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bd2eb-8108-4457-8005-165460438f46 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.337002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1103/2176] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:51.341023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613376c2-0a63-4e79-906d-50560fb43a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.341577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613376c2-0a63-4e79-906d-50560fb43a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.341946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613376c2-0a63-4e79-906d-50560fb43a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.342225 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613376c2-0a63-4e79-906d-50560fb43a4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.345790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613376c2-0a63-4e79-906d-50560fb43a4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.345949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ae158-4611-4ca7-9636-47f92b6d1f8c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.346042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613376c2-0a63-4e79-906d-50560fb43a4d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.356810 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1074/2177] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:51.362947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613376c2-0a63-4e79-906d-50560fb43a4d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.364810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ae158-4611-4ca7-9636-47f92b6d1f8c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.370809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1b41f-6b9d-4a79-b2df-75887444ad89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.371496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1b41f-6b9d-4a79-b2df-75887444ad89 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.371757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1b41f-6b9d-4a79-b2df-75887444ad89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.372036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1b41f-6b9d-4a79-b2df-75887444ad89 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.372764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1b41f-6b9d-4a79-b2df-75887444ad89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.374811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f1b41f-6b9d-4a79-b2df-75887444ad89 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.375336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613376c2-0a63-4e79-906d-50560fb43a4d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.380763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb7d0e-2fd4-4f5e-923c-48f7df002054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.381403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb7d0e-2fd4-4f5e-923c-48f7df002054 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.381680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb7d0e-2fd4-4f5e-923c-48f7df002054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.382015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb7d0e-2fd4-4f5e-923c-48f7df002054 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.382692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb7d0e-2fd4-4f5e-923c-48f7df002054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.384290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb7d0e-2fd4-4f5e-923c-48f7df002054 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.400548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1075/2178] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:51.402441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb7d0e-2fd4-4f5e-923c-48f7df002054 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.408725 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1104/2179] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:51 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:51.413081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1b41f-6b9d-4a79-b2df-75887444ad89 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.417298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb7d0e-2fd4-4f5e-923c-48f7df002054 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.484121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1b41f-6b9d-4a79-b2df-75887444ad89 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.485770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250d0286-35b1-4840-b7f3-9bddaa1cbda9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.486659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0286-35b1-4840-b7f3-9bddaa1cbda9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.486949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0286-35b1-4840-b7f3-9bddaa1cbda9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.487227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250d0286-35b1-4840-b7f3-9bddaa1cbda9 admin admin] PATH_INFO: `/v3/projects/e29dfb4148ff4076a2e42df6b2460aa5/users/150235a48e694c8584f4ac521510c80a/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.492205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65bd39-7a37-4e98-9052-7b1d749c5449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.492205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65bd39-7a37-4e98-9052-7b1d749c5449 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.492205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65bd39-7a37-4e98-9052-7b1d749c5449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.492205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65bd39-7a37-4e98-9052-7b1d749c5449 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.492682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65bd39-7a37-4e98-9052-7b1d749c5449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.492934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65bd39-7a37-4e98-9052-7b1d749c5449 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.509806 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d0286-35b1-4840-b7f3-9bddaa1cbda9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e29dfb4148ff4076a2e42df6b2460aa5, user_id=150235a48e694c8584f4ac521510c80a, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.512685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250d0286-35b1-4840-b7f3-9bddaa1cbda9 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.518221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1105/2180] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:51 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:51.522655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65bd39-7a37-4e98-9052-7b1d749c5449 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.525212 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:51.533849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1076/2181] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:51 2022] PUT /identity/v3/projects/e29dfb4148ff4076a2e42df6b2460aa5/users/150235a48e694c8584f4ac521510c80a/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:51.544123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375431f7-6d3f-4872-b96c-68c74effce75 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.549826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375431f7-6d3f-4872-b96c-68c74effce75 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.550303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375431f7-6d3f-4872-b96c-68c74effce75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.551497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65bd39-7a37-4e98-9052-7b1d749c5449 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.613547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-375431f7-6d3f-4872-b96c-68c74effce75 None None] MFA Rules not processed for user `150235a48e694c8584f4ac521510c80a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:51.613547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-375431f7-6d3f-4872-b96c-68c74effce75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:51.620229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a877439-759a-45ae-9363-4859d915921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.620945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a877439-759a-45ae-9363-4859d915921f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.621275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a877439-759a-45ae-9363-4859d915921f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.621641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a877439-759a-45ae-9363-4859d915921f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.622427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a877439-759a-45ae-9363-4859d915921f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.624202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a877439-759a-45ae-9363-4859d915921f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.625512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:51.637781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1106/2182] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:51.640416 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a877439-759a-45ae-9363-4859d915921f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.650539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a877439-759a-45ae-9363-4859d915921f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.655555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf2eaf-fee4-4142-aec0-e3c9e1b6ed00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.656238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf2eaf-fee4-4142-aec0-e3c9e1b6ed00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.656522 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf2eaf-fee4-4142-aec0-e3c9e1b6ed00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.656800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf2eaf-fee4-4142-aec0-e3c9e1b6ed00 admin admin] PATH_INFO: `/v3/projects/9ab58a6a058b42f993da458d850e76dc/users/84e883e399654efea8e9b226e8b22e93/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.676808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf2eaf-fee4-4142-aec0-e3c9e1b6ed00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ab58a6a058b42f993da458d850e76dc, user_id=84e883e399654efea8e9b226e8b22e93, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.680576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf2eaf-fee4-4142-aec0-e3c9e1b6ed00 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.683709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1077/2183] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:51.697276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375431f7-6d3f-4872-b96c-68c74effce75 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.698688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-375431f7-6d3f-4872-b96c-68c74effce75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:51.701882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1107/2184] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:51 2022] PUT /identity/v3/projects/9ab58a6a058b42f993da458d850e76dc/users/84e883e399654efea8e9b226e8b22e93/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:51.706040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf2eaf-fee4-4142-aec0-e3c9e1b6ed00 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.766281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375431f7-6d3f-4872-b96c-68c74effce75 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.798761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf2eaf-fee4-4142-aec0-e3c9e1b6ed00 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.806948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea459c-4c8d-47ed-9409-e40ada81160e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.808812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea459c-4c8d-47ed-9409-e40ada81160e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.809579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea459c-4c8d-47ed-9409-e40ada81160e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.810348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ea459c-4c8d-47ed-9409-e40ada81160e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.812021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea459c-4c8d-47ed-9409-e40ada81160e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.815394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ea459c-4c8d-47ed-9409-e40ada81160e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.826541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c83eab-2723-4283-9046-478d88494772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.827343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c83eab-2723-4283-9046-478d88494772 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.827833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c83eab-2723-4283-9046-478d88494772 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.828188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c83eab-2723-4283-9046-478d88494772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.842573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1108/2185] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:51.846902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea459c-4c8d-47ed-9409-e40ada81160e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:51.859015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ea459c-4c8d-47ed-9409-e40ada81160e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:51.865382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed669ac2-8dd8-48c4-849f-f788be5a713e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:51.866664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed669ac2-8dd8-48c4-849f-f788be5a713e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.866868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed669ac2-8dd8-48c4-849f-f788be5a713e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.867104 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed669ac2-8dd8-48c4-849f-f788be5a713e admin admin] PATH_INFO: `/v3/projects/9ab58a6a058b42f993da458d850e76dc/users/84e883e399654efea8e9b226e8b22e93/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.869439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed669ac2-8dd8-48c4-849f-f788be5a713e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ab58a6a058b42f993da458d850e76dc, user_id=84e883e399654efea8e9b226e8b22e93, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.872456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed669ac2-8dd8-48c4-849f-f788be5a713e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.884886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1109/2186] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:51 2022] PUT /identity/v3/projects/9ab58a6a058b42f993da458d850e76dc/users/84e883e399654efea8e9b226e8b22e93/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:26:51.896739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec5736-0487-4943-ac70-4eb52a6c5369 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:51.897083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec5736-0487-4943-ac70-4eb52a6c5369 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:51.897318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec5736-0487-4943-ac70-4eb52a6c5369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:51.926326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c83eab-2723-4283-9046-478d88494772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:51.927911 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c83eab-2723-4283-9046-478d88494772 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:51.938953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:51.943348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1078/2187] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:51.973264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-82ec5736-0487-4943-ac70-4eb52a6c5369 None None] MFA Rules not processed for user `84e883e399654efea8e9b226e8b22e93`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:51.978567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-82ec5736-0487-4943-ac70-4eb52a6c5369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:52.058421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1110/2188] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:52.070793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c83eab-2723-4283-9046-478d88494772 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:52.072431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a8c83eab-2723-4283-9046-478d88494772 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:52.094437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c83eab-2723-4283-9046-478d88494772 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:52.162030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856fc35-058a-4e26-a9b7-b50cb2f4f769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:52.162795 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856fc35-058a-4e26-a9b7-b50cb2f4f769 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:52.163160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856fc35-058a-4e26-a9b7-b50cb2f4f769 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:52.163533 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856fc35-058a-4e26-a9b7-b50cb2f4f769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:52.175712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856fc35-058a-4e26-a9b7-b50cb2f4f769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:52.177978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856fc35-058a-4e26-a9b7-b50cb2f4f769 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:52.196147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1079/2189] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:52.978361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec5736-0487-4943-ac70-4eb52a6c5369 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:53.030768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:53.054914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec5736-0487-4943-ac70-4eb52a6c5369 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:53.059939 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:53.060602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:53.060882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:53.061157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] PATH_INFO: `/v3/users/a6559e42d56843ffa9b1645237129ebd` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:53.062481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6559e42d56843ffa9b1645237129ebd)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:53.064987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:53.116060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6559e42d56843ffa9b1645237129ebd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:53.117056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6559e42d56843ffa9b1645237129ebd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:53.130014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6559e42d56843ffa9b1645237129ebd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:53.138046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6559e42d56843ffa9b1645237129ebd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:53.147326 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6559e42d56843ffa9b1645237129ebd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:53.153682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1111/2190] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:52 2022] DELETE /identity/v3/users/a6559e42d56843ffa9b1645237129ebd => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:53.158742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eef7aa-ed88-4969-8c67-8f0f22e772ea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:53.159357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eef7aa-ed88-4969-8c67-8f0f22e772ea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:53.159357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eef7aa-ed88-4969-8c67-8f0f22e772ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:53.181477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-92eef7aa-ed88-4969-8c67-8f0f22e772ea None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:53.182909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-92eef7aa-ed88-4969-8c67-8f0f22e772ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:53.252711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1080/2191] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:53.263481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:53.265521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:53.330979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88102d24-41b6-4980-8557-85471bb4fa7c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:53.335300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d302f8-e6bc-4df1-ae21-d324408e680b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:53.335990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d302f8-e6bc-4df1-ae21-d324408e680b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:53.336517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d302f8-e6bc-4df1-ae21-d324408e680b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:53.336836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d302f8-e6bc-4df1-ae21-d324408e680b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:53.350414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d302f8-e6bc-4df1-ae21-d324408e680b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:53.351943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d302f8-e6bc-4df1-ae21-d324408e680b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:53.363996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1112/2192] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:26:53.453105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1675a5c-6ed7-42e6-a8d6-a997a884bf3f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:53.453389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1675a5c-6ed7-42e6-a8d6-a997a884bf3f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:53.453673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1675a5c-6ed7-42e6-a8d6-a997a884bf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:53.487308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f1675a5c-6ed7-42e6-a8d6-a997a884bf3f None None] MFA Rules not processed for user `150235a48e694c8584f4ac521510c80a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:53.489382 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f1675a5c-6ed7-42e6-a8d6-a997a884bf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:53.582978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1081/2193] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:26:53.723514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d302f8-e6bc-4df1-ae21-d324408e680b service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:53.725050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-99d302f8-e6bc-4df1-ae21-d324408e680b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:53.742506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d302f8-e6bc-4df1-ae21-d324408e680b service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:53.752911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:53.753720 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:53.754190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:53.754541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] PATH_INFO: `/v3/projects/3bf8703dfd3f4b0e97874b0520607d10` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:53.755846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf8703dfd3f4b0e97874b0520607d10)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:53.758069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:53.818447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf8703dfd3f4b0e97874b0520607d10', 'request_id': 'req-876cc98f-c388-4248-a9c8-18066a638ccc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:53.825127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf8703dfd3f4b0e97874b0520607d10', 'request_id': 'req-876cc98f-c388-4248-a9c8-18066a638ccc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:53.827295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:53.832444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf8703dfd3f4b0e97874b0520607d10', 'request_id': 'req-876cc98f-c388-4248-a9c8-18066a638ccc'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:53.850064 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] The token cache is being invalidate because project 3bf8703dfd3f4b0e97874b0520607d10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:53.850337 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:53.850990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:53.852774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1113/2194] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:53 2022] DELETE /identity/v3/projects/3bf8703dfd3f4b0e97874b0520607d10 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:53.856881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1675a5c-6ed7-42e6-a8d6-a997a884bf3f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:53.868428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1675a5c-6ed7-42e6-a8d6-a997a884bf3f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:53.927687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:53.941132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:53.941849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:53.942127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:53.942399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] PATH_INFO: `/v3/users/84e883e399654efea8e9b226e8b22e93` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:53.943585 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e883e399654efea8e9b226e8b22e93)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:53.945521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:54.001473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e883e399654efea8e9b226e8b22e93'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:54.004027 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e883e399654efea8e9b226e8b22e93'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:54.022113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e883e399654efea8e9b226e8b22e93'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:54.028884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e883e399654efea8e9b226e8b22e93'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:54.055419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f4092145-3bc2-42a3-8095-02597eb6a72d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e883e399654efea8e9b226e8b22e93'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:54.066356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1082/2195] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:26:53 2022] DELETE /identity/v3/users/84e883e399654efea8e9b226e8b22e93 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:54.376547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:54.378123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:54.406284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876cc98f-c388-4248-a9c8-18066a638ccc admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:54.474308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:54.474966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:54.475243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:54.475514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] PATH_INFO: `/v3/projects/9ab58a6a058b42f993da458d850e76dc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:54.476715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ab58a6a058b42f993da458d850e76dc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:54.478706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:54.530289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ab58a6a058b42f993da458d850e76dc', 'request_id': 'req-9b2cbcf1-3549-4093-954b-f48200b964fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:54.536770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ab58a6a058b42f993da458d850e76dc', 'request_id': 'req-9b2cbcf1-3549-4093-954b-f48200b964fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:54.542990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ab58a6a058b42f993da458d850e76dc', 'request_id': 'req-9b2cbcf1-3549-4093-954b-f48200b964fd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:54.557818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] The token cache is being invalidate because project 9ab58a6a058b42f993da458d850e76dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:26:54.558106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:54.558730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b2cbcf1-3549-4093-954b-f48200b964fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:26:54.560247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1114/2196] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:26:54 2022] DELETE /identity/v3/projects/9ab58a6a058b42f993da458d850e76dc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:54.565255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792765a6-f28b-4a18-817e-2c702a4e50c9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:54.565784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792765a6-f28b-4a18-817e-2c702a4e50c9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:54.565880 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792765a6-f28b-4a18-817e-2c702a4e50c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:54.586541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-792765a6-f28b-4a18-817e-2c702a4e50c9 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:54.587637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-792765a6-f28b-4a18-817e-2c702a4e50c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:54.656786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1083/2197] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:54.665931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8c464-e0a1-4e2c-aa5b-dc8a6c63f9e2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:54.667273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8c464-e0a1-4e2c-aa5b-dc8a6c63f9e2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:54.667756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8c464-e0a1-4e2c-aa5b-dc8a6c63f9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:54.692105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b6b8c464-e0a1-4e2c-aa5b-dc8a6c63f9e2 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:54.693296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b6b8c464-e0a1-4e2c-aa5b-dc8a6c63f9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:54.702206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1115/2198] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:54.706952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792765a6-f28b-4a18-817e-2c702a4e50c9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:54.718136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792765a6-f28b-4a18-817e-2c702a4e50c9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:54.723621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0d6f7-6f59-4a9e-be18-cd0509178ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:54.724250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0d6f7-6f59-4a9e-be18-cd0509178ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:54.724522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0d6f7-6f59-4a9e-be18-cd0509178ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:54.724796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0d6f7-6f59-4a9e-be18-cd0509178ba3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:54.725496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0d6f7-6f59-4a9e-be18-cd0509178ba3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:26:54.725826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0d6f7-6f59-4a9e-be18-cd0509178ba3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:54.727404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0d6f7-6f59-4a9e-be18-cd0509178ba3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:54.739769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1084/2199] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:26:54.744475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8c464-e0a1-4e2c-aa5b-dc8a6c63f9e2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:54.756072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8c464-e0a1-4e2c-aa5b-dc8a6c63f9e2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:54.761915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6645f46-bab9-4cc4-8952-d9623d553222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:54.762577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6645f46-bab9-4cc4-8952-d9623d553222 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:54.762858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6645f46-bab9-4cc4-8952-d9623d553222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:54.763112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6645f46-bab9-4cc4-8952-d9623d553222 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:54.763781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6645f46-bab9-4cc4-8952-d9623d553222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:54.765793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6645f46-bab9-4cc4-8952-d9623d553222 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:54.803284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1116/2200] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:26:54 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:26:54.807653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0d6f7-6f59-4a9e-be18-cd0509178ba3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:54.818226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0d6f7-6f59-4a9e-be18-cd0509178ba3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:54.871721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:54.880424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bd0d9-f33e-4ce5-a28e-49143870f2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:54.881125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bd0d9-f33e-4ce5-a28e-49143870f2bf admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:54.881421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bd0d9-f33e-4ce5-a28e-49143870f2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:54.881697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487bd0d9-f33e-4ce5-a28e-49143870f2bf admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:54.882492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487bd0d9-f33e-4ce5-a28e-49143870f2bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:54.884724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487bd0d9-f33e-4ce5-a28e-49143870f2bf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:54.914349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1085/2201] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:26:54 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:26:54.915687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6645f46-bab9-4cc4-8952-d9623d553222 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:54.926831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6645f46-bab9-4cc4-8952-d9623d553222 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:54.932657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c49672-1f13-4257-bba5-3afd2901f941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:54.933290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c49672-1f13-4257-bba5-3afd2901f941 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:54.933573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c49672-1f13-4257-bba5-3afd2901f941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:54.933881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c49672-1f13-4257-bba5-3afd2901f941 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:54.934633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c49672-1f13-4257-bba5-3afd2901f941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:54.936220 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c49672-1f13-4257-bba5-3afd2901f941 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:54.947439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1117/2202] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:26:54.952534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bd0d9-f33e-4ce5-a28e-49143870f2bf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:54.964049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487bd0d9-f33e-4ce5-a28e-49143870f2bf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:54.970465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fcbec-5aa7-4e61-be9a-ff8425723e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:54.971131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fcbec-5aa7-4e61-be9a-ff8425723e72 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:54.971414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fcbec-5aa7-4e61-be9a-ff8425723e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:54.971695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fcbec-5aa7-4e61-be9a-ff8425723e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:54.972393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fcbec-5aa7-4e61-be9a-ff8425723e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:54.974063 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fcbec-5aa7-4e61-be9a-ff8425723e72 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:54.986231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1086/2203] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:26:54.990781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c49672-1f13-4257-bba5-3afd2901f941 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:55.001268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c49672-1f13-4257-bba5-3afd2901f941 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:55.006084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ec17b-87dc-41ba-b063-d509534747a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:55.006749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ec17b-87dc-41ba-b063-d509534747a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:55.007022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ec17b-87dc-41ba-b063-d509534747a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:55.007293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ec17b-87dc-41ba-b063-d509534747a4 admin admin] PATH_INFO: `/v3/projects/94262684b6f94054b16b2c4ce4187265/users/ee036a1d7e0447369c3fd712e2000271/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:55.022861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ec17b-87dc-41ba-b063-d509534747a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94262684b6f94054b16b2c4ce4187265, user_id=ee036a1d7e0447369c3fd712e2000271, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:55.025557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ec17b-87dc-41ba-b063-d509534747a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:55.040160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1118/2204] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:26:54 2022] PUT /identity/v3/projects/94262684b6f94054b16b2c4ce4187265/users/ee036a1d7e0447369c3fd712e2000271/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:26:55.050790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443e427-798a-43eb-a793-ab8fd0523394 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:55.051082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443e427-798a-43eb-a793-ab8fd0523394 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:55.051361 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443e427-798a-43eb-a793-ab8fd0523394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:55.087355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4443e427-798a-43eb-a793-ab8fd0523394 None None] MFA Rules not processed for user `ee036a1d7e0447369c3fd712e2000271`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:26:55.089118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4443e427-798a-43eb-a793-ab8fd0523394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:55.161706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1087/2205] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:26:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:26:55.173041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ec17b-87dc-41ba-b063-d509534747a4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:26:55.175021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-230ec17b-87dc-41ba-b063-d509534747a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:26:55.190220 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:26:55.246233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ec17b-87dc-41ba-b063-d509534747a4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:26:55.250105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3dae8-ccf8-4529-928a-db3a6518e46b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:26:55.250739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3dae8-ccf8-4529-928a-db3a6518e46b service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:26:55.251016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3dae8-ccf8-4529-928a-db3a6518e46b service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:26:55.251288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3dae8-ccf8-4529-928a-db3a6518e46b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:26:55.259928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3dae8-ccf8-4529-928a-db3a6518e46b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:26:55.261420 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f3dae8-ccf8-4529-928a-db3a6518e46b service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:26:55.270697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1119/2206] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:26:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:27:03.841509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443e427-798a-43eb-a793-ab8fd0523394 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:03.911666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443e427-798a-43eb-a793-ab8fd0523394 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:03.917309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:03.918444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:03.918964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:03.919356 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] PATH_INFO: `/v3/users/ee036a1d7e0447369c3fd712e2000271` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:03.920639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee036a1d7e0447369c3fd712e2000271)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:03.922978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:03.982364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee036a1d7e0447369c3fd712e2000271'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:03.983161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee036a1d7e0447369c3fd712e2000271'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:03.995832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee036a1d7e0447369c3fd712e2000271'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:03.997200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:04.006889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee036a1d7e0447369c3fd712e2000271'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:04.014531 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee036a1d7e0447369c3fd712e2000271'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:04.020719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1088/2207] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:27:03 2022] DELETE /identity/v3/users/ee036a1d7e0447369c3fd712e2000271 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:04.025528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a3704-e7c0-4702-a65e-ca7db8f8df75 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:04.026117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a3704-e7c0-4702-a65e-ca7db8f8df75 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:04.026219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a3704-e7c0-4702-a65e-ca7db8f8df75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:04.046359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0e0a3704-e7c0-4702-a65e-ca7db8f8df75 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:04.047576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e0a3704-e7c0-4702-a65e-ca7db8f8df75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:04.112655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1120/2208] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:27:04.123053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:04.124665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:04.183162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90991211-5527-4b2e-a43b-2c641ac38e51 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:04.187420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2d2f5-573f-47da-861f-831ce86057e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:04.188048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2d2f5-573f-47da-861f-831ce86057e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:04.188316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2d2f5-573f-47da-861f-831ce86057e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:04.188590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2d2f5-573f-47da-861f-831ce86057e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:04.198717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2d2f5-573f-47da-861f-831ce86057e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:04.200140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2d2f5-573f-47da-861f-831ce86057e3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:04.210207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1089/2209] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:27:04.539295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a3704-e7c0-4702-a65e-ca7db8f8df75 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:04.541472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0e0a3704-e7c0-4702-a65e-ca7db8f8df75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:04.554435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a3704-e7c0-4702-a65e-ca7db8f8df75 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:04.559026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:04.559689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:04.559974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:04.560257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] PATH_INFO: `/v3/projects/94262684b6f94054b16b2c4ce4187265` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:04.561465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] RBAC: Authorizing `identity:delete_project(project_id=94262684b6f94054b16b2c4ce4187265)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:04.563572 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:04.631494 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94262684b6f94054b16b2c4ce4187265', 'request_id': 'req-4a566553-3ecb-43d3-8ea5-19d238c8877b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:04.638335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94262684b6f94054b16b2c4ce4187265', 'request_id': 'req-4a566553-3ecb-43d3-8ea5-19d238c8877b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:04.647508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94262684b6f94054b16b2c4ce4187265', 'request_id': 'req-4a566553-3ecb-43d3-8ea5-19d238c8877b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:04.669534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] The token cache is being invalidate because project 94262684b6f94054b16b2c4ce4187265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:27:04.669857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:04.670586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4a566553-3ecb-43d3-8ea5-19d238c8877b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:04.672265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1121/2210] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:27:04 2022] DELETE /identity/v3/projects/94262684b6f94054b16b2c4ce4187265 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:04.677753 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94019fe4-b227-4c62-a17b-988133f4e7d0 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:04.678083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94019fe4-b227-4c62-a17b-988133f4e7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:04.678404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94019fe4-b227-4c62-a17b-988133f4e7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:04.698991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-94019fe4-b227-4c62-a17b-988133f4e7d0 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:04.700199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-94019fe4-b227-4c62-a17b-988133f4e7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:04.764824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1090/2211] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:04.773675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321db616-6c29-411c-97a8-62065a842103 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:04.773994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321db616-6c29-411c-97a8-62065a842103 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:04.774274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321db616-6c29-411c-97a8-62065a842103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:04.795680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-321db616-6c29-411c-97a8-62065a842103 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:04.796808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-321db616-6c29-411c-97a8-62065a842103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:04.805378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1122/2212] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:04.809892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94019fe4-b227-4c62-a17b-988133f4e7d0 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:04.820539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94019fe4-b227-4c62-a17b-988133f4e7d0 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:04.825267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b1f95-f6ea-45df-96a6-32f66fe792b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:04.826042 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b1f95-f6ea-45df-96a6-32f66fe792b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:04.826396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b1f95-f6ea-45df-96a6-32f66fe792b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:04.826706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362b1f95-f6ea-45df-96a6-32f66fe792b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:04.827395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b1f95-f6ea-45df-96a6-32f66fe792b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:27:04.827887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b1f95-f6ea-45df-96a6-32f66fe792b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:04.829874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362b1f95-f6ea-45df-96a6-32f66fe792b0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:04.847639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1091/2213] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:27:04.853333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321db616-6c29-411c-97a8-62065a842103 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:04.863001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321db616-6c29-411c-97a8-62065a842103 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:04.867816 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7b1d6-2097-45c2-b733-ac76c6e0fb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:04.868453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7b1d6-2097-45c2-b733-ac76c6e0fb28 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:04.868735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7b1d6-2097-45c2-b733-ac76c6e0fb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:04.869010 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e7b1d6-2097-45c2-b733-ac76c6e0fb28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:04.869683 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7b1d6-2097-45c2-b733-ac76c6e0fb28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:04.871655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e7b1d6-2097-45c2-b733-ac76c6e0fb28 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:04.902242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1123/2214] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:27:04 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:27:04.906769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b1f95-f6ea-45df-96a6-32f66fe792b0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:04.917275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362b1f95-f6ea-45df-96a6-32f66fe792b0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:04.985661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e8203-04f8-4af5-bcb7-634dd0c3af46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:04.986328 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e8203-04f8-4af5-bcb7-634dd0c3af46 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:04.986616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e8203-04f8-4af5-bcb7-634dd0c3af46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:04.986921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e8203-04f8-4af5-bcb7-634dd0c3af46 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:04.987654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e8203-04f8-4af5-bcb7-634dd0c3af46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:04.989619 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e8203-04f8-4af5-bcb7-634dd0c3af46 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:05.013123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1092/2215] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:27:04 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:27:05.017598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7b1d6-2097-45c2-b733-ac76c6e0fb28 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:05.028311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e7b1d6-2097-45c2-b733-ac76c6e0fb28 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:05.034041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca79c5-54c9-4dcd-ac7e-5d65dd59d789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:05.034679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca79c5-54c9-4dcd-ac7e-5d65dd59d789 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:05.034982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca79c5-54c9-4dcd-ac7e-5d65dd59d789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:05.035253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca79c5-54c9-4dcd-ac7e-5d65dd59d789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:05.035955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca79c5-54c9-4dcd-ac7e-5d65dd59d789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:05.037655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca79c5-54c9-4dcd-ac7e-5d65dd59d789 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:05.040317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:05.049676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1124/2216] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:27:05.054144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e8203-04f8-4af5-bcb7-634dd0c3af46 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:05.059883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:05.066560 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e8203-04f8-4af5-bcb7-634dd0c3af46 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:05.074908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acffff8-794c-4ff1-957f-7c57e0c12d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:05.075593 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acffff8-794c-4ff1-957f-7c57e0c12d8c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:05.075881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acffff8-794c-4ff1-957f-7c57e0c12d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:05.076159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acffff8-794c-4ff1-957f-7c57e0c12d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:05.076894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acffff8-794c-4ff1-957f-7c57e0c12d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:05.078552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acffff8-794c-4ff1-957f-7c57e0c12d8c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:05.089892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1093/2217] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:27:05.094595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca79c5-54c9-4dcd-ac7e-5d65dd59d789 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:05.104839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca79c5-54c9-4dcd-ac7e-5d65dd59d789 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:05.109927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201d68d-25a0-4648-9b20-ad9313131896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:05.110621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d68d-25a0-4648-9b20-ad9313131896 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:05.110904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d68d-25a0-4648-9b20-ad9313131896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:05.111215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d68d-25a0-4648-9b20-ad9313131896 admin admin] PATH_INFO: `/v3/projects/c9cbf9b8d2e646eb945deb24d55fa0ad/users/7a4c3b33511646b59d89dc933827dcb5/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:05.128586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201d68d-25a0-4648-9b20-ad9313131896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9cbf9b8d2e646eb945deb24d55fa0ad, user_id=7a4c3b33511646b59d89dc933827dcb5, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:05.131321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201d68d-25a0-4648-9b20-ad9313131896 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:05.145662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1125/2218] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:27:05 2022] PUT /identity/v3/projects/c9cbf9b8d2e646eb945deb24d55fa0ad/users/7a4c3b33511646b59d89dc933827dcb5/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:05.151079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a8df1-7730-45ae-a2a5-6322b6f2f69d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:05.151421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a8df1-7730-45ae-a2a5-6322b6f2f69d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:05.151700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a8df1-7730-45ae-a2a5-6322b6f2f69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:05.171746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-552a8df1-7730-45ae-a2a5-6322b6f2f69d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:05.172980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-552a8df1-7730-45ae-a2a5-6322b6f2f69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:05.238189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1094/2219] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:27:05.248405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201d68d-25a0-4648-9b20-ad9313131896 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:05.250631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4201d68d-25a0-4648-9b20-ad9313131896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:05.312290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201d68d-25a0-4648-9b20-ad9313131896 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:05.316288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acd6f1-6b66-4469-baef-a1cb33bb45cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:05.316936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acd6f1-6b66-4469-baef-a1cb33bb45cd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:05.317214 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acd6f1-6b66-4469-baef-a1cb33bb45cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:05.317513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37acd6f1-6b66-4469-baef-a1cb33bb45cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:05.327566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acd6f1-6b66-4469-baef-a1cb33bb45cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:05.329063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37acd6f1-6b66-4469-baef-a1cb33bb45cd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:05.342602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1126/2220] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:27:06.336686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd01f7-b5ee-4b9d-8374-134224d1b425 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:06.337194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd01f7-b5ee-4b9d-8374-134224d1b425 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:06.337194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd01f7-b5ee-4b9d-8374-134224d1b425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:06.395026 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-50fd01f7-b5ee-4b9d-8374-134224d1b425 None None] MFA Rules not processed for user `7a4c3b33511646b59d89dc933827dcb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:06.397466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-50fd01f7-b5ee-4b9d-8374-134224d1b425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:06.483988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1095/2221] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:06 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:06.496248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acd6f1-6b66-4469-baef-a1cb33bb45cd service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:06.497909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-37acd6f1-6b66-4469-baef-a1cb33bb45cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:06.611317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37acd6f1-6b66-4469-baef-a1cb33bb45cd service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:06.619294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0f73d-f2da-4fa3-9482-4ee5cb7b2068 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:06.620261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0f73d-f2da-4fa3-9482-4ee5cb7b2068 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:06.620948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0f73d-f2da-4fa3-9482-4ee5cb7b2068 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:06.621404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0f73d-f2da-4fa3-9482-4ee5cb7b2068 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:06.631532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0f73d-f2da-4fa3-9482-4ee5cb7b2068 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:06.633328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0f73d-f2da-4fa3-9482-4ee5cb7b2068 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:06.655544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1127/2222] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:27:06 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:27:08.040163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e694-8984-4423-9d2f-40bac55786b1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:08.040737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e694-8984-4423-9d2f-40bac55786b1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:08.040737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff8e694-8984-4423-9d2f-40bac55786b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:08.067433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8ff8e694-8984-4423-9d2f-40bac55786b1 None None] MFA Rules not processed for user `7a4c3b33511646b59d89dc933827dcb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:08.068660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8ff8e694-8984-4423-9d2f-40bac55786b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:08.076179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1096/2223] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:08 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:27:16.642012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:18.054620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=66 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:22.536977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0f73d-f2da-4fa3-9482-4ee5cb7b2068 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:22.539136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b1b0f73d-f2da-4fa3-9482-4ee5cb7b2068 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:22.553765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0f73d-f2da-4fa3-9482-4ee5cb7b2068 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:22.558811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:22.559456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:22.559738 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:22.560013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] PATH_INFO: `/v3/users/16b884a9f6a8444c8f06e0fd5ef909b2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:22.561277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16b884a9f6a8444c8f06e0fd5ef909b2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:22.563258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:22.621365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16b884a9f6a8444c8f06e0fd5ef909b2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:22.622327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16b884a9f6a8444c8f06e0fd5ef909b2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:22.639810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16b884a9f6a8444c8f06e0fd5ef909b2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:22.649108 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16b884a9f6a8444c8f06e0fd5ef909b2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:22.659268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16b884a9f6a8444c8f06e0fd5ef909b2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:22.666373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1128/2224] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:27:22 2022] DELETE /identity/v3/users/16b884a9f6a8444c8f06e0fd5ef909b2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:22.671311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f7cf-1298-4d2e-a720-2ca8caa8b201 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:22.671840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f7cf-1298-4d2e-a720-2ca8caa8b201 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:22.671960 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f7cf-1298-4d2e-a720-2ca8caa8b201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:22.691465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c6b9f7cf-1298-4d2e-a720-2ca8caa8b201 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:22.692721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6b9f7cf-1298-4d2e-a720-2ca8caa8b201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:22.760172 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1097/2225] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:22.767108 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:22.769025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:22.825161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f33c3-a3ee-4d5e-9303-4c5066418740 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:22.829086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44db4457-f0ab-4fd6-9cc2-8866c883e8a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:22.830106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db4457-f0ab-4fd6-9cc2-8866c883e8a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:22.830106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db4457-f0ab-4fd6-9cc2-8866c883e8a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:22.830312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db4457-f0ab-4fd6-9cc2-8866c883e8a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:22.840006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db4457-f0ab-4fd6-9cc2-8866c883e8a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:22.841428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44db4457-f0ab-4fd6-9cc2-8866c883e8a6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:22.851561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1129/2226] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:27:23.241610 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9f7cf-1298-4d2e-a720-2ca8caa8b201 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:23.243230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c6b9f7cf-1298-4d2e-a720-2ca8caa8b201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:23.256197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9f7cf-1298-4d2e-a720-2ca8caa8b201 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:23.262656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:23.263509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:23.264000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:23.264500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] PATH_INFO: `/v3/projects/9cdde04ed4324908bd55bd38bcb36f9f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:23.266581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cdde04ed4324908bd55bd38bcb36f9f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:23.269150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:23.362432 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cdde04ed4324908bd55bd38bcb36f9f', 'request_id': 'req-144cc4bc-307b-4d27-9bcd-07f2c21f278d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:23.372939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cdde04ed4324908bd55bd38bcb36f9f', 'request_id': 'req-144cc4bc-307b-4d27-9bcd-07f2c21f278d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:23.387250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cdde04ed4324908bd55bd38bcb36f9f', 'request_id': 'req-144cc4bc-307b-4d27-9bcd-07f2c21f278d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:23.402978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] The token cache is being invalidate because project 9cdde04ed4324908bd55bd38bcb36f9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:27:23.403487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:23.406872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-144cc4bc-307b-4d27-9bcd-07f2c21f278d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:23.409131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1098/2227] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:27:23 2022] DELETE /identity/v3/projects/9cdde04ed4324908bd55bd38bcb36f9f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:23.417531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775fe15-2f29-49df-a77f-efbbd383f830 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:23.417844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775fe15-2f29-49df-a77f-efbbd383f830 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:23.418286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775fe15-2f29-49df-a77f-efbbd383f830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:23.443945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b775fe15-2f29-49df-a77f-efbbd383f830 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:23.445170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b775fe15-2f29-49df-a77f-efbbd383f830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:23.515762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1130/2228] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:27:23.524228 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94c792-f4b2-4e84-9fa3-f5ea993789e8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:23.524631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94c792-f4b2-4e84-9fa3-f5ea993789e8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:23.525138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b94c792-f4b2-4e84-9fa3-f5ea993789e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:23.547529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5b94c792-f4b2-4e84-9fa3-f5ea993789e8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:23.548922 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5b94c792-f4b2-4e84-9fa3-f5ea993789e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:23.559051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1099/2229] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:23.563677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775fe15-2f29-49df-a77f-efbbd383f830 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:23.574861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775fe15-2f29-49df-a77f-efbbd383f830 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:23.580437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb266a-373e-4b69-b05e-d5b07168ec79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:23.580763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb266a-373e-4b69-b05e-d5b07168ec79 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:23.581043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb266a-373e-4b69-b05e-d5b07168ec79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:23.581319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb266a-373e-4b69-b05e-d5b07168ec79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:23.582066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb266a-373e-4b69-b05e-d5b07168ec79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:27:23.582377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb266a-373e-4b69-b05e-d5b07168ec79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:23.584002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb266a-373e-4b69-b05e-d5b07168ec79 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:23.602308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1131/2230] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:27:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:27:23.606514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94c792-f4b2-4e84-9fa3-f5ea993789e8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:23.618782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b94c792-f4b2-4e84-9fa3-f5ea993789e8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:23.626618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5339610-0485-4b69-a794-c9f846854b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:23.627650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5339610-0485-4b69-a794-c9f846854b9d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:23.628120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5339610-0485-4b69-a794-c9f846854b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:23.628592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5339610-0485-4b69-a794-c9f846854b9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:23.629454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5339610-0485-4b69-a794-c9f846854b9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:23.631491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5339610-0485-4b69-a794-c9f846854b9d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:23.668495 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1100/2231] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:27:23 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:27:23.673137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb266a-373e-4b69-b05e-d5b07168ec79 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:23.678758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:23.685490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb266a-373e-4b69-b05e-d5b07168ec79 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:23.748533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa582c5f-873e-4786-8532-06752f9e08c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:23.749209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa582c5f-873e-4786-8532-06752f9e08c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:23.749493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa582c5f-873e-4786-8532-06752f9e08c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:23.749909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa582c5f-873e-4786-8532-06752f9e08c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:23.750702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa582c5f-873e-4786-8532-06752f9e08c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:23.752648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa582c5f-873e-4786-8532-06752f9e08c2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:23.775607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1132/2232] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:27:23 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:27:23.779984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5339610-0485-4b69-a794-c9f846854b9d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:23.790858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5339610-0485-4b69-a794-c9f846854b9d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:23.795802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce77bc-a8d7-4790-a773-c34d61af8341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:23.796408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce77bc-a8d7-4790-a773-c34d61af8341 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:23.796760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce77bc-a8d7-4790-a773-c34d61af8341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:23.796916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce77bc-a8d7-4790-a773-c34d61af8341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:23.797648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce77bc-a8d7-4790-a773-c34d61af8341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:23.799298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bce77bc-a8d7-4790-a773-c34d61af8341 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:23.810322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1101/2233] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:27:23.814388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa582c5f-873e-4786-8532-06752f9e08c2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:23.825032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa582c5f-873e-4786-8532-06752f9e08c2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:23.829729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba4123c-23c8-444f-ad9a-c9fcc999521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:23.830391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4123c-23c8-444f-ad9a-c9fcc999521a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:23.830673 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4123c-23c8-444f-ad9a-c9fcc999521a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:23.830947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4123c-23c8-444f-ad9a-c9fcc999521a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:23.831686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4123c-23c8-444f-ad9a-c9fcc999521a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:23.833313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba4123c-23c8-444f-ad9a-c9fcc999521a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:23.844970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1133/2234] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:27:23.849296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce77bc-a8d7-4790-a773-c34d61af8341 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:23.861657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce77bc-a8d7-4790-a773-c34d61af8341 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:23.867851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b310b82-dc03-47ae-9701-db77d423d45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:23.868640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b310b82-dc03-47ae-9701-db77d423d45b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:23.868998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b310b82-dc03-47ae-9701-db77d423d45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:23.869343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b310b82-dc03-47ae-9701-db77d423d45b admin admin] PATH_INFO: `/v3/projects/6fc12ef80ee841f5af4bc2d56966cadc/users/c394476504af4c1d8bea462a5113cebc/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:23.887412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b310b82-dc03-47ae-9701-db77d423d45b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc12ef80ee841f5af4bc2d56966cadc, user_id=c394476504af4c1d8bea462a5113cebc, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:23.890297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b310b82-dc03-47ae-9701-db77d423d45b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:23.905711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1102/2235] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:27:23 2022] PUT /identity/v3/projects/6fc12ef80ee841f5af4bc2d56966cadc/users/c394476504af4c1d8bea462a5113cebc/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:23.911008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1403e26-5faa-4260-9bfa-cbda07dd18d8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:23.911306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1403e26-5faa-4260-9bfa-cbda07dd18d8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:23.911584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1403e26-5faa-4260-9bfa-cbda07dd18d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:23.931565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e1403e26-5faa-4260-9bfa-cbda07dd18d8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:23.932972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e1403e26-5faa-4260-9bfa-cbda07dd18d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:23.993885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1134/2236] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:24.005636 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b310b82-dc03-47ae-9701-db77d423d45b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:24.007476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7b310b82-dc03-47ae-9701-db77d423d45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:24.031558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:24.072748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b310b82-dc03-47ae-9701-db77d423d45b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:24.077147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b3c63-d2b3-4874-869e-767beb94618e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:24.077946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b3c63-d2b3-4874-869e-767beb94618e service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:24.078233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b3c63-d2b3-4874-869e-767beb94618e service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:24.078513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b3c63-d2b3-4874-869e-767beb94618e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:24.088835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b3c63-d2b3-4874-869e-767beb94618e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:24.090558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238b3c63-d2b3-4874-869e-767beb94618e service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:24.101320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1103/2237] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:27:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:27:24.845092 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a2a2b-6e2c-4b0f-951a-a96096ffb0e6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:24.845678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a2a2b-6e2c-4b0f-951a-a96096ffb0e6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:24.845678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a2a2b-6e2c-4b0f-951a-a96096ffb0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:24.885340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2c9a2a2b-6e2c-4b0f-951a-a96096ffb0e6 None None] MFA Rules not processed for user `c394476504af4c1d8bea462a5113cebc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:24.886740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2c9a2a2b-6e2c-4b0f-951a-a96096ffb0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:24.961991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1135/2238] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:24.973731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b3c63-d2b3-4874-869e-767beb94618e service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:24.975292 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-238b3c63-d2b3-4874-869e-767beb94618e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:25.040308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238b3c63-d2b3-4874-869e-767beb94618e service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:25.044242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b94db8-74e0-490b-9953-26008ba2c5d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:25.044870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b94db8-74e0-490b-9953-26008ba2c5d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:25.045153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b94db8-74e0-490b-9953-26008ba2c5d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:25.045473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b94db8-74e0-490b-9953-26008ba2c5d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:25.055227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b94db8-74e0-490b-9953-26008ba2c5d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:25.056655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b94db8-74e0-490b-9953-26008ba2c5d4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:25.066825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1104/2239] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:27:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:27:27.539483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e210f2a-8396-41e9-9f55-7a55b2b056cb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:27.540055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e210f2a-8396-41e9-9f55-7a55b2b056cb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:27.540055 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e210f2a-8396-41e9-9f55-7a55b2b056cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:27.595845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-7e210f2a-8396-41e9-9f55-7a55b2b056cb None None] MFA Rules not processed for user `c394476504af4c1d8bea462a5113cebc`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:27.597688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e210f2a-8396-41e9-9f55-7a55b2b056cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:27.611283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1136/2240] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:27:29.720508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7536d34-ba93-446e-a3f1-25e0874d7143 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:29.721493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7536d34-ba93-446e-a3f1-25e0874d7143 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:29.723143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7536d34-ba93-446e-a3f1-25e0874d7143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:29.756970 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e7536d34-ba93-446e-a3f1-25e0874d7143 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:29.762633 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7536d34-ba93-446e-a3f1-25e0874d7143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:29.833093 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1105/2241] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:27:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:27:29.893424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e210f2a-8396-41e9-9f55-7a55b2b056cb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:29.894855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e210f2a-8396-41e9-9f55-7a55b2b056cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:29.902383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:29.973523 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e210f2a-8396-41e9-9f55-7a55b2b056cb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:29.979068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20067350-d4d2-4e25-a1a2-3d4a9a66fb19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:29.979719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20067350-d4d2-4e25-a1a2-3d4a9a66fb19 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:29.980003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20067350-d4d2-4e25-a1a2-3d4a9a66fb19 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:29.980282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20067350-d4d2-4e25-a1a2-3d4a9a66fb19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:29.990155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20067350-d4d2-4e25-a1a2-3d4a9a66fb19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:29.992011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20067350-d4d2-4e25-a1a2-3d4a9a66fb19 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:30.002078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1137/2242] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:27:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:27:31.002998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25468b5e-1885-4a5a-b2cb-115dfe12b4e4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:31.004059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25468b5e-1885-4a5a-b2cb-115dfe12b4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:31.004840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25468b5e-1885-4a5a-b2cb-115dfe12b4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:31.052944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-25468b5e-1885-4a5a-b2cb-115dfe12b4e4 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:31.054320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-25468b5e-1885-4a5a-b2cb-115dfe12b4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:31.064048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1106/2243] 10.222.0.39 () {58 vars in 967 bytes} [Tue Jul 26 07:27:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:31.079136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20067350-d4d2-4e25-a1a2-3d4a9a66fb19 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:31.093893 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20067350-d4d2-4e25-a1a2-3d4a9a66fb19 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:31.099807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4565928-9100-4c39-80ab-b23beaeafe82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:31.100472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565928-9100-4c39-80ab-b23beaeafe82 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:31.100749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565928-9100-4c39-80ab-b23beaeafe82 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:31.101014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4565928-9100-4c39-80ab-b23beaeafe82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:31.110280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4565928-9100-4c39-80ab-b23beaeafe82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:31.111899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4565928-9100-4c39-80ab-b23beaeafe82 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:31.123208 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1138/2244] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:27:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:27:31.348590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca3d8-2bbc-415e-a94e-dd9825f5be3e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:31.349661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca3d8-2bbc-415e-a94e-dd9825f5be3e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:31.350513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ca3d8-2bbc-415e-a94e-dd9825f5be3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:31.395241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e97ca3d8-2bbc-415e-a94e-dd9825f5be3e None None] MFA Rules not processed for user `7a4c3b33511646b59d89dc933827dcb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:31.397179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e97ca3d8-2bbc-415e-a94e-dd9825f5be3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:31.559820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1107/2245] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:31 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:31.593001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c9a87-f5d4-42ea-9cfc-60bce1eacf15 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:31.593634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c9a87-f5d4-42ea-9cfc-60bce1eacf15 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:31.594062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c9a87-f5d4-42ea-9cfc-60bce1eacf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:31.638124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9b8c9a87-f5d4-42ea-9cfc-60bce1eacf15 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:31.641452 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9b8c9a87-f5d4-42ea-9cfc-60bce1eacf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:31.677289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1139/2246] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:27:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:31.680600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ca3d8-2bbc-415e-a94e-dd9825f5be3e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:31.695340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ca3d8-2bbc-415e-a94e-dd9825f5be3e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:31.701557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e4949-9a66-4998-b969-99bf40c744d0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:31.702862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e4949-9a66-4998-b969-99bf40c744d0 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:31.703196 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e4949-9a66-4998-b969-99bf40c744d0 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:31.703580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840e4949-9a66-4998-b969-99bf40c744d0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:31.712326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:31.715327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e4949-9a66-4998-b969-99bf40c744d0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:31.717788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840e4949-9a66-4998-b969-99bf40c744d0 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:31.729597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1108/2247] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:27:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:27:32.377446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94add-530f-4843-b58b-3a5d3a0f9691 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:32.378151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94add-530f-4843-b58b-3a5d3a0f9691 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:32.378151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b94add-530f-4843-b58b-3a5d3a0f9691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:32.398931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-74b94add-530f-4843-b58b-3a5d3a0f9691 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:32.400188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-74b94add-530f-4843-b58b-3a5d3a0f9691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:32.408179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1140/2248] 10.222.0.39 () {58 vars in 967 bytes} [Tue Jul 26 07:27:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:32.422167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e4949-9a66-4998-b969-99bf40c744d0 service swift] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:32.433863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840e4949-9a66-4998-b969-99bf40c744d0 service swift] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:32.439664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecb38e-0911-419d-a4c0-02758e389c41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:32.440319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecb38e-0911-419d-a4c0-02758e389c41 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:32.440598 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecb38e-0911-419d-a4c0-02758e389c41 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:32.440871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecb38e-0911-419d-a4c0-02758e389c41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:32.450394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecb38e-0911-419d-a4c0-02758e389c41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:32.452100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecb38e-0911-419d-a4c0-02758e389c41 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:32.461454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1109/2249] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:27:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:27:32.659223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b94add-530f-4843-b58b-3a5d3a0f9691 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:32.663004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-74b94add-530f-4843-b58b-3a5d3a0f9691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:32.692756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b94add-530f-4843-b58b-3a5d3a0f9691 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:32.708080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:32.708774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:32.709048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:32.715701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] PATH_INFO: `/v3/users/8896a709512b43ed8963d78cee158878` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:32.717951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8896a709512b43ed8963d78cee158878)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:32.720524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:32.822877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8896a709512b43ed8963d78cee158878'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:32.834050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8896a709512b43ed8963d78cee158878'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:32.872314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8896a709512b43ed8963d78cee158878'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:32.885948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8896a709512b43ed8963d78cee158878'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:32.909379 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8896a709512b43ed8963d78cee158878'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:32.930177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1141/2250] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:27:32 2022] DELETE /identity/v3/users/8896a709512b43ed8963d78cee158878 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:32.940726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63204ee5-e26d-4651-b29e-23c2ba0f45c4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:32.941413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63204ee5-e26d-4651-b29e-23c2ba0f45c4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:32.942115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63204ee5-e26d-4651-b29e-23c2ba0f45c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:32.996047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-63204ee5-e26d-4651-b29e-23c2ba0f45c4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:33.005284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-63204ee5-e26d-4651-b29e-23c2ba0f45c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:33.125662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1110/2251] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:33.140321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:33.141838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:33.202819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:33.223807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea137f9-80e2-4979-9044-0719cdd7fa2d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:33.229373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9823bc-0f55-49fa-9e0b-366854bba5cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:33.230166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9823bc-0f55-49fa-9e0b-366854bba5cd service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:33.230961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9823bc-0f55-49fa-9e0b-366854bba5cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:33.231630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9823bc-0f55-49fa-9e0b-366854bba5cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:33.244945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9823bc-0f55-49fa-9e0b-366854bba5cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:33.247384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9823bc-0f55-49fa-9e0b-366854bba5cd service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:33.279628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1142/2252] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:27:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:27:33.823287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63204ee5-e26d-4651-b29e-23c2ba0f45c4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:33.825037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-63204ee5-e26d-4651-b29e-23c2ba0f45c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:33.839707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63204ee5-e26d-4651-b29e-23c2ba0f45c4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:33.845769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:33.846776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:33.847296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:33.847634 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] PATH_INFO: `/v3/projects/008f2ea45e5e4ef3995ad89621819ad7` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:33.850976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] RBAC: Authorizing `identity:delete_project(project_id=008f2ea45e5e4ef3995ad89621819ad7)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:33.853396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:33.921439 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '008f2ea45e5e4ef3995ad89621819ad7', 'request_id': 'req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:33.932700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '008f2ea45e5e4ef3995ad89621819ad7', 'request_id': 'req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:33.943544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '008f2ea45e5e4ef3995ad89621819ad7', 'request_id': 'req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:33.963261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] The token cache is being invalidate because project 008f2ea45e5e4ef3995ad89621819ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:27:33.963642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:33.964428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7ab17517-c2eb-4e7d-8a07-53c7b120e67a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:33.967583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1111/2253] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:27:33 2022] DELETE /identity/v3/projects/008f2ea45e5e4ef3995ad89621819ad7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:33.987047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7effc-ac63-4964-b260-71560e331f22 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:33.987047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7effc-ac63-4964-b260-71560e331f22 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:33.987047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7effc-ac63-4964-b260-71560e331f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:34.015032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-55e7effc-ac63-4964-b260-71560e331f22 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:34.016396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55e7effc-ac63-4964-b260-71560e331f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:34.108154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1143/2254] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:34.117760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4938a03-5bec-46e0-aa42-54cc89d1be87 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:34.118033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4938a03-5bec-46e0-aa42-54cc89d1be87 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:34.118410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4938a03-5bec-46e0-aa42-54cc89d1be87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:34.143919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a4938a03-5bec-46e0-aa42-54cc89d1be87 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:34.146213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a4938a03-5bec-46e0-aa42-54cc89d1be87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:34.160676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1112/2255] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:34.166298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7effc-ac63-4964-b260-71560e331f22 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:34.182822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7effc-ac63-4964-b260-71560e331f22 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:34.195012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07eb533-365d-4c6f-ba30-2b22958c6ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:34.196051 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07eb533-365d-4c6f-ba30-2b22958c6ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:34.196753 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07eb533-365d-4c6f-ba30-2b22958c6ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:34.197228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07eb533-365d-4c6f-ba30-2b22958c6ad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:34.198663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07eb533-365d-4c6f-ba30-2b22958c6ad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:27:34.199205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07eb533-365d-4c6f-ba30-2b22958c6ad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:34.202345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07eb533-365d-4c6f-ba30-2b22958c6ad4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:34.220597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1144/2256] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:27:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:27:34.225407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4938a03-5bec-46e0-aa42-54cc89d1be87 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:34.237694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4938a03-5bec-46e0-aa42-54cc89d1be87 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:34.245925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca6a57-ec63-4e3e-bdd3-98d548c7c2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:34.247207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6a57-ec63-4e3e-bdd3-98d548c7c2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:34.247696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6a57-ec63-4e3e-bdd3-98d548c7c2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:34.248234 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca6a57-ec63-4e3e-bdd3-98d548c7c2d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:34.249691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca6a57-ec63-4e3e-bdd3-98d548c7c2d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:34.252976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca6a57-ec63-4e3e-bdd3-98d548c7c2d6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:34.298086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1113/2257] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:27:34 2022] POST /identity/v3/projects => generated 399 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:27:34.302609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07eb533-365d-4c6f-ba30-2b22958c6ad4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:34.319875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07eb533-365d-4c6f-ba30-2b22958c6ad4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:34.408367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d94f85-a82d-4bc2-b318-662d69480487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:34.409440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d94f85-a82d-4bc2-b318-662d69480487 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:34.409929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d94f85-a82d-4bc2-b318-662d69480487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:34.410297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d94f85-a82d-4bc2-b318-662d69480487 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:34.411627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d94f85-a82d-4bc2-b318-662d69480487 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:34.417535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d94f85-a82d-4bc2-b318-662d69480487 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:34.463201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1145/2258] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:27:34 2022] POST /identity/v3/users => generated 300 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:27:34.468926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca6a57-ec63-4e3e-bdd3-98d548c7c2d6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:34.481830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca6a57-ec63-4e3e-bdd3-98d548c7c2d6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:34.489561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40840f30-2ffb-4ccd-a946-c93fe328ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:34.490991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40840f30-2ffb-4ccd-a946-c93fe328ae25 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:34.491466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40840f30-2ffb-4ccd-a946-c93fe328ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:34.491935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40840f30-2ffb-4ccd-a946-c93fe328ae25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:34.493215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40840f30-2ffb-4ccd-a946-c93fe328ae25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:34.496245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40840f30-2ffb-4ccd-a946-c93fe328ae25 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:34.514693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1114/2259] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:27:34.519436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d94f85-a82d-4bc2-b318-662d69480487 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:34.532739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d94f85-a82d-4bc2-b318-662d69480487 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:34.543435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c07799-af50-4370-8e3f-f5764eb45b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:34.544639 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c07799-af50-4370-8e3f-f5764eb45b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:34.545158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c07799-af50-4370-8e3f-f5764eb45b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:34.545561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c07799-af50-4370-8e3f-f5764eb45b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:34.546793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c07799-af50-4370-8e3f-f5764eb45b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:34.552227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c07799-af50-4370-8e3f-f5764eb45b1c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:34.599691 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1146/2260] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:27:34.601774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40840f30-2ffb-4ccd-a946-c93fe328ae25 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:34.639249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40840f30-2ffb-4ccd-a946-c93fe328ae25 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:34.654600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be20dd-e487-43f4-a743-2c4fba33de4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:34.654600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be20dd-e487-43f4-a743-2c4fba33de4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:34.654600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be20dd-e487-43f4-a743-2c4fba33de4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:34.655065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be20dd-e487-43f4-a743-2c4fba33de4d admin admin] PATH_INFO: `/v3/projects/384e462d5bd041498ad6f160ffd59385/users/b4e88eb05bee4306aa72d875b5d64b95/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:34.688717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be20dd-e487-43f4-a743-2c4fba33de4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=384e462d5bd041498ad6f160ffd59385, user_id=b4e88eb05bee4306aa72d875b5d64b95, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:34.694530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be20dd-e487-43f4-a743-2c4fba33de4d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:34.720264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1115/2261] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:27:34 2022] PUT /identity/v3/projects/384e462d5bd041498ad6f160ffd59385/users/b4e88eb05bee4306aa72d875b5d64b95/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:34.738381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e406e52-93fd-410e-8651-0352fe6afa10 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:34.739289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e406e52-93fd-410e-8651-0352fe6afa10 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:34.739777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e406e52-93fd-410e-8651-0352fe6afa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:34.794677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1e406e52-93fd-410e-8651-0352fe6afa10 None None] MFA Rules not processed for user `b4e88eb05bee4306aa72d875b5d64b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:34.796409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1e406e52-93fd-410e-8651-0352fe6afa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:34.844113 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:34.883418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1147/2262] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:34 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:34.896223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be20dd-e487-43f4-a743-2c4fba33de4d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:34.898256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-36be20dd-e487-43f4-a743-2c4fba33de4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:34.906364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:35.004492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be20dd-e487-43f4-a743-2c4fba33de4d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:35.008915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010a239-b2e8-4442-9ba9-9f1857f952aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:35.009597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010a239-b2e8-4442-9ba9-9f1857f952aa service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:35.009912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010a239-b2e8-4442-9ba9-9f1857f952aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:35.010187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010a239-b2e8-4442-9ba9-9f1857f952aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:35.021070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010a239-b2e8-4442-9ba9-9f1857f952aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:35.022623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010a239-b2e8-4442-9ba9-9f1857f952aa service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:35.033371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1116/2263] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:27:34 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:27:35.247836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e406e52-93fd-410e-8651-0352fe6afa10 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:35.250159 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1e406e52-93fd-410e-8651-0352fe6afa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:35.344468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e406e52-93fd-410e-8651-0352fe6afa10 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:35.352925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:35.354056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:35.354539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:35.355016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] PATH_INFO: `/v3/users/150235a48e694c8584f4ac521510c80a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:35.357273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=150235a48e694c8584f4ac521510c80a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:35.361535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:35.465037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '150235a48e694c8584f4ac521510c80a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:35.466234 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '150235a48e694c8584f4ac521510c80a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:35.493854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '150235a48e694c8584f4ac521510c80a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:35.509495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '150235a48e694c8584f4ac521510c80a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:35.529131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '150235a48e694c8584f4ac521510c80a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:35.540329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1148/2264] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:27:35 2022] DELETE /identity/v3/users/150235a48e694c8584f4ac521510c80a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:35.545167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ac157-3f53-4fe2-9965-c26015eee23a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:35.545473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ac157-3f53-4fe2-9965-c26015eee23a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:35.545811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ac157-3f53-4fe2-9965-c26015eee23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:35.572080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e71ac157-3f53-4fe2-9965-c26015eee23a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:35.573412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e71ac157-3f53-4fe2-9965-c26015eee23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:35.666161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1117/2265] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:27:35.676681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:35.678529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:35.746134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11d2f8-7b4b-4c07-95e0-7cf98c52ae39 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:35.750123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c5691f-dbb2-4427-a48e-b02f64951384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:35.750753 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5691f-dbb2-4427-a48e-b02f64951384 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:35.751027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5691f-dbb2-4427-a48e-b02f64951384 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:35.751302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c5691f-dbb2-4427-a48e-b02f64951384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:35.761694 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5691f-dbb2-4427-a48e-b02f64951384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:35.763236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c5691f-dbb2-4427-a48e-b02f64951384 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:35.768983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:35.775838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1149/2266] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:27:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:27:36.015177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ac157-3f53-4fe2-9965-c26015eee23a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:36.017257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e71ac157-3f53-4fe2-9965-c26015eee23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:36.041316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ac157-3f53-4fe2-9965-c26015eee23a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:36.049027 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:36.049998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:36.050787 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:36.051233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] PATH_INFO: `/v3/projects/e29dfb4148ff4076a2e42df6b2460aa5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:36.053422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e29dfb4148ff4076a2e42df6b2460aa5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:36.059607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:36.130426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e29dfb4148ff4076a2e42df6b2460aa5', 'request_id': 'req-9a414888-27a5-4f8b-b760-2e1fd844bc50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:36.137794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e29dfb4148ff4076a2e42df6b2460aa5', 'request_id': 'req-9a414888-27a5-4f8b-b760-2e1fd844bc50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:36.148412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e29dfb4148ff4076a2e42df6b2460aa5', 'request_id': 'req-9a414888-27a5-4f8b-b760-2e1fd844bc50'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:36.163485 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] The token cache is being invalidate because project e29dfb4148ff4076a2e42df6b2460aa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:27:36.163738 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:36.164416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:36.166003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1118/2267] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:27:36 2022] DELETE /identity/v3/projects/e29dfb4148ff4076a2e42df6b2460aa5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:27:38.969056 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de8e64-89a3-44c8-9baa-b73c20fc3e4d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:38.969681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de8e64-89a3-44c8-9baa-b73c20fc3e4d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:38.970041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7de8e64-89a3-44c8-9baa-b73c20fc3e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:39.001033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d7de8e64-89a3-44c8-9baa-b73c20fc3e4d None None] MFA Rules not processed for user `57029844165f4d83892176fe82594685`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:39.002388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7de8e64-89a3-44c8-9baa-b73c20fc3e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:39.067808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1150/2268] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:42.677705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:42.679267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:42.717056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:42.759597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414888-27a5-4f8b-b760-2e1fd844bc50 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:42.765480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:42.766472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:42.766851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:42.767404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] PATH_INFO: `/v3/users/b4e88eb05bee4306aa72d875b5d64b95` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:42.769333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4e88eb05bee4306aa72d875b5d64b95)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:42.773177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:42.889025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4e88eb05bee4306aa72d875b5d64b95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:42.890480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4e88eb05bee4306aa72d875b5d64b95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:42.919375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4e88eb05bee4306aa72d875b5d64b95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:42.942886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4e88eb05bee4306aa72d875b5d64b95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:42.974329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4e88eb05bee4306aa72d875b5d64b95'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:42.983530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1119/2269] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:27:42 2022] DELETE /identity/v3/users/b4e88eb05bee4306aa72d875b5d64b95 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:42.988783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cafe9-4d31-4f64-b55d-e945a029c6ce None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:42.996999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cafe9-4d31-4f64-b55d-e945a029c6ce None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:42.998628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00cafe9-4d31-4f64-b55d-e945a029c6ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:43.078711 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c00cafe9-4d31-4f64-b55d-e945a029c6ce None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:43.082111 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c00cafe9-4d31-4f64-b55d-e945a029c6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:43.276330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1151/2270] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:43.325657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:43.327434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:43.421594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd9591-4411-477b-b07b-1258439a0f1b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:43.432474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089b4e8d-eb97-4b0a-857c-afc5f3dbf963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:43.435024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4e8d-eb97-4b0a-857c-afc5f3dbf963 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:43.435711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4e8d-eb97-4b0a-857c-afc5f3dbf963 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:43.436242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089b4e8d-eb97-4b0a-857c-afc5f3dbf963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:43.490994 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b4e8d-eb97-4b0a-857c-afc5f3dbf963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:43.492525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089b4e8d-eb97-4b0a-857c-afc5f3dbf963 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:43.506947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1120/2271] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:27:44.985499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cafe9-4d31-4f64-b55d-e945a029c6ce None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:44.989338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c00cafe9-4d31-4f64-b55d-e945a029c6ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:45.012688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00cafe9-4d31-4f64-b55d-e945a029c6ce None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:45.020093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:45.021150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:45.021708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:45.022270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] PATH_INFO: `/v3/projects/384e462d5bd041498ad6f160ffd59385` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:45.024437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=384e462d5bd041498ad6f160ffd59385)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:45.026820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:45.111294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384e462d5bd041498ad6f160ffd59385', 'request_id': 'req-331384d8-a77f-461d-8366-19732c661f01'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:45.130325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384e462d5bd041498ad6f160ffd59385', 'request_id': 'req-331384d8-a77f-461d-8366-19732c661f01'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:45.139317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384e462d5bd041498ad6f160ffd59385', 'request_id': 'req-331384d8-a77f-461d-8366-19732c661f01'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:45.171041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] The token cache is being invalidate because project 384e462d5bd041498ad6f160ffd59385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:27:45.171527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:45.172754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:27:45.175374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1152/2272] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:27:44 2022] DELETE /identity/v3/projects/384e462d5bd041498ad6f160ffd59385 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:27:45.673343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a86851-8fb1-4be3-8d3c-4f442442a229 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:45.674306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a86851-8fb1-4be3-8d3c-4f442442a229 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:45.674922 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a86851-8fb1-4be3-8d3c-4f442442a229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:45.709545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-91a86851-8fb1-4be3-8d3c-4f442442a229 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:45.711685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-91a86851-8fb1-4be3-8d3c-4f442442a229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:45.779393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:45.795323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1121/2273] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:27:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:27:45.811477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:27:45.813241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:45.864798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:27:45.950843 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331384d8-a77f-461d-8366-19732c661f01 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:27:45.958703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f92ef-41ee-454a-8d76-9c98cefab9c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:27:45.959696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f92ef-41ee-454a-8d76-9c98cefab9c5 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:45.960225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f92ef-41ee-454a-8d76-9c98cefab9c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:45.960598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f92ef-41ee-454a-8d76-9c98cefab9c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:45.974789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f92ef-41ee-454a-8d76-9c98cefab9c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:27:45.976796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f92ef-41ee-454a-8d76-9c98cefab9c5 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:27:45.988872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1153/2274] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:27:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:27:53.214169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e844f6-90f9-4ce8-a073-a13a583c8572 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:27:53.215231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e844f6-90f9-4ce8-a073-a13a583c8572 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:27:53.215736 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e844f6-90f9-4ce8-a073-a13a583c8572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:27:53.252306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c5e844f6-90f9-4ce8-a073-a13a583c8572 None None] MFA Rules not processed for user `886151888fc84ccf890d609ba62da9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:27:53.255084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c5e844f6-90f9-4ce8-a073-a13a583c8572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:27:53.372408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1122/2275] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:27:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:27:55.983923 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=19 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:28:03.360081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=27 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:28:09.267659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62654d83-c518-4da5-86ac-0101cda8b49c None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:09.268289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62654d83-c518-4da5-86ac-0101cda8b49c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:09.268606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62654d83-c518-4da5-86ac-0101cda8b49c None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:09.272664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1154/2276] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:28:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:28:09.282997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db31e81-e065-482c-8ccd-23baad7b2cf7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:09.283943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db31e81-e065-482c-8ccd-23baad7b2cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:09.284120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db31e81-e065-482c-8ccd-23baad7b2cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:09.323423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-8db31e81-e065-482c-8ccd-23baad7b2cf7 None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:28:09.327420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8db31e81-e065-482c-8ccd-23baad7b2cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:09.489437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1123/2277] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:28:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:28:09.511152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62654d83-c518-4da5-86ac-0101cda8b49c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:09.514271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-62654d83-c518-4da5-86ac-0101cda8b49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:09.609345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62654d83-c518-4da5-86ac-0101cda8b49c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:09.613892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc63bc-3961-48fd-9f55-8d02913d5cd7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c6943686c843e588df14f7c6dffb8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:09.614783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc63bc-3961-48fd-9f55-8d02913d5cd7 service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:09.615124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc63bc-3961-48fd-9f55-8d02913d5cd7 service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:09.615478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc63bc-3961-48fd-9f55-8d02913d5cd7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:09.628902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc63bc-3961-48fd-9f55-8d02913d5cd7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:09.630819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc63bc-3961-48fd-9f55-8d02913d5cd7 service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:09.645123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1155/2278] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:28:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:28:09.757673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbd166-aee1-4c07-9570-3326c3c8c739 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:09.758217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbd166-aee1-4c07-9570-3326c3c8c739 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:09.758749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbd166-aee1-4c07-9570-3326c3c8c739 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:09.762681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1124/2279] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:28:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:28:09.771829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ff060-5a4f-4a0a-9883-7d6b8123a475 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:09.772307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ff060-5a4f-4a0a-9883-7d6b8123a475 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:09.772838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ff060-5a4f-4a0a-9883-7d6b8123a475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:09.807048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f76ff060-5a4f-4a0a-9883-7d6b8123a475 None None] MFA Rules not processed for user `1911678a39f749939dc8476617a884fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:28:09.809004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f76ff060-5a4f-4a0a-9883-7d6b8123a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:09.891617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1156/2280] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:28:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:28:09.906046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbd166-aee1-4c07-9570-3326c3c8c739 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:09.909195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-01bbd166-aee1-4c07-9570-3326c3c8c739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:09.930940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbd166-aee1-4c07-9570-3326c3c8c739 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:09.935892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662e8f5-53aa-44e9-b592-bb989f78f728 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:09.936809 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662e8f5-53aa-44e9-b592-bb989f78f728 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:09.937185 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662e8f5-53aa-44e9-b592-bb989f78f728 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:09.937553 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662e8f5-53aa-44e9-b592-bb989f78f728 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:09.950061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662e8f5-53aa-44e9-b592-bb989f78f728 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:09.952726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662e8f5-53aa-44e9-b592-bb989f78f728 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:09.967549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1125/2281] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:28:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:28:10.083714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f535f-670c-4fe5-9cc3-d513119bef08 None None] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:10.084154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f535f-670c-4fe5-9cc3-d513119bef08 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:10.084568 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785f535f-670c-4fe5-9cc3-d513119bef08 None None] PATH_INFO: `/` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:10.086507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1157/2282] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:28:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:28:10.093458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c1a4b-6cf4-4052-93d3-f2d7e5019e55 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:10.093983 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c1a4b-6cf4-4052-93d3-f2d7e5019e55 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:10.094408 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c1a4b-6cf4-4052-93d3-f2d7e5019e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:10.116689 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9d3c1a4b-6cf4-4052-93d3-f2d7e5019e55 None None] MFA Rules not processed for user `6d50427014064f099af87932c77a849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:28:10.118081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9d3c1a4b-6cf4-4052-93d3-f2d7e5019e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:10.131971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1126/2283] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:28:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:28:10.144658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f535f-670c-4fe5-9cc3-d513119bef08 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:10.163191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785f535f-670c-4fe5-9cc3-d513119bef08 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:10.167696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62324d57-ae3a-4019-9675-df6274c0fcfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:10.168472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62324d57-ae3a-4019-9675-df6274c0fcfb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:10.169021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62324d57-ae3a-4019-9675-df6274c0fcfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:10.169524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62324d57-ae3a-4019-9675-df6274c0fcfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:10.187701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62324d57-ae3a-4019-9675-df6274c0fcfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:10.189335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62324d57-ae3a-4019-9675-df6274c0fcfb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:10.203801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1158/2284] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:28:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:28:14.618249 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9de17b-19fa-440e-bd1d-d844936418bb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:14.618865 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9de17b-19fa-440e-bd1d-d844936418bb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:14.618992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9de17b-19fa-440e-bd1d-d844936418bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:14.641638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2e9de17b-19fa-440e-bd1d-d844936418bb None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:28:14.642954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2e9de17b-19fa-440e-bd1d-d844936418bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:14.651326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1127/2285] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:28:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:28:14.667073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62324d57-ae3a-4019-9675-df6274c0fcfb service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:14.686119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62324d57-ae3a-4019-9675-df6274c0fcfb service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:14.691852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b799b-8ed3-439b-90b8-b2a578147821 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:14.692569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b799b-8ed3-439b-90b8-b2a578147821 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:14.692904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b799b-8ed3-439b-90b8-b2a578147821 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:14.693231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794b799b-8ed3-439b-90b8-b2a578147821 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:14.703292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b799b-8ed3-439b-90b8-b2a578147821 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:14.704956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794b799b-8ed3-439b-90b8-b2a578147821 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:14.715542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1159/2286] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:28:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:28:24.276115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25545a-93fe-432a-8d04-14c1f9e0f392 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:24.276927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25545a-93fe-432a-8d04-14c1f9e0f392 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:24.276927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25545a-93fe-432a-8d04-14c1f9e0f392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:24.338319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0e25545a-93fe-432a-8d04-14c1f9e0f392 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:28:24.339261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e25545a-93fe-432a-8d04-14c1f9e0f392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:24.362048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1128/2287] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 07:28:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:28:24.394009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b799b-8ed3-439b-90b8-b2a578147821 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:24.418978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794b799b-8ed3-439b-90b8-b2a578147821 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:24.430917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bb16c-6bf0-4203-aef5-a4b95a2cd6dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:24.432239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bb16c-6bf0-4203-aef5-a4b95a2cd6dd service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:24.433057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bb16c-6bf0-4203-aef5-a4b95a2cd6dd service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:24.433776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898bb16c-6bf0-4203-aef5-a4b95a2cd6dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:24.453160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bb16c-6bf0-4203-aef5-a4b95a2cd6dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:24.456929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898bb16c-6bf0-4203-aef5-a4b95a2cd6dd service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:24.476486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1160/2288] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:28:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:28:34.311189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=46 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:28:34.468138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=67 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:28:37.800596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e79442-29c4-4da6-be2b-5ad895117eee None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:37.801255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e79442-29c4-4da6-be2b-5ad895117eee None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:37.801255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e79442-29c4-4da6-be2b-5ad895117eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:37.827009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-19e79442-29c4-4da6-be2b-5ad895117eee None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:28:37.829118 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-19e79442-29c4-4da6-be2b-5ad895117eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:37.841852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1129/2289] 10.222.0.39 () {58 vars in 967 bytes} [Tue Jul 26 07:28:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:28:37.856679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bb16c-6bf0-4203-aef5-a4b95a2cd6dd service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:37.869958 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898bb16c-6bf0-4203-aef5-a4b95a2cd6dd service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:37.877363 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70828f-1ede-4ee1-aef0-104cb5cde17c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:37.878140 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70828f-1ede-4ee1-aef0-104cb5cde17c service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:37.878420 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70828f-1ede-4ee1-aef0-104cb5cde17c service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:37.878695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70828f-1ede-4ee1-aef0-104cb5cde17c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:37.890227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70828f-1ede-4ee1-aef0-104cb5cde17c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:37.892520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc70828f-1ede-4ee1-aef0-104cb5cde17c service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:37.904609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1161/2290] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:28:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 07:28:40.188247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57a1f7-8c7b-4e76-b9b9-e8f0e0e332c8 None None] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:40.189099 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57a1f7-8c7b-4e76-b9b9-e8f0e0e332c8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:40.189402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57a1f7-8c7b-4e76-b9b9-e8f0e0e332c8 None None] PATH_INFO: `/` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:40.191319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1130/2291] 10.222.0.39 () {54 vars in 921 bytes} [Tue Jul 26 07:28:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 26 07:28:40.256103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413997-449a-4d7b-a6b3-e9a489bf27c8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:40.257076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413997-449a-4d7b-a6b3-e9a489bf27c8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:40.257428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75413997-449a-4d7b-a6b3-e9a489bf27c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:40.292549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-75413997-449a-4d7b-a6b3-e9a489bf27c8 None None] MFA Rules not processed for user `51a5093a60ef45919ab6e159adf58336`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:28:40.294709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-75413997-449a-4d7b-a6b3-e9a489bf27c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:40.339288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57a1f7-8c7b-4e76-b9b9-e8f0e0e332c8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:40.343539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e57a1f7-8c7b-4e76-b9b9-e8f0e0e332c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:40.439899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1162/2292] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Jul 26 07:28:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:28:40.454928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413997-449a-4d7b-a6b3-e9a489bf27c8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:40.456729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-75413997-449a-4d7b-a6b3-e9a489bf27c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:40.472279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75413997-449a-4d7b-a6b3-e9a489bf27c8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:40.474277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57a1f7-8c7b-4e76-b9b9-e8f0e0e332c8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:40.478734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81daa5c-f3ec-42e9-84d2-ccefd0ba7a7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:40.479518 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81daa5c-f3ec-42e9-84d2-ccefd0ba7a7b service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:40.479865 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81daa5c-f3ec-42e9-84d2-ccefd0ba7a7b service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:40.480206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81daa5c-f3ec-42e9-84d2-ccefd0ba7a7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:40.481692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:40.482609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:40.482926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:40.483260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] PATH_INFO: `/v3/users/7a4c3b33511646b59d89dc933827dcb5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:40.484583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a4c3b33511646b59d89dc933827dcb5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:40.487332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:40.490782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81daa5c-f3ec-42e9-84d2-ccefd0ba7a7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:40.492580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81daa5c-f3ec-42e9-84d2-ccefd0ba7a7b service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:40.504990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1163/2293] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:28:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:28:40.561325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a4c3b33511646b59d89dc933827dcb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:28:40.562559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a4c3b33511646b59d89dc933827dcb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:28:40.583568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a4c3b33511646b59d89dc933827dcb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:28:40.593997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a4c3b33511646b59d89dc933827dcb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:28:40.604061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3e2d4500-0d31-46e4-8089-88f69a72fd2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a4c3b33511646b59d89dc933827dcb5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:28:40.611653 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1131/2294] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:28:40 2022] DELETE /identity/v3/users/7a4c3b33511646b59d89dc933827dcb5 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:28:40.839132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81daa5c-f3ec-42e9-84d2-ccefd0ba7a7b service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:40.853927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81daa5c-f3ec-42e9-84d2-ccefd0ba7a7b service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:40.936735 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:40.937379 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:40.937665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:40.937973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] PATH_INFO: `/v3/projects/c9cbf9b8d2e646eb945deb24d55fa0ad` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:40.939220 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9cbf9b8d2e646eb945deb24d55fa0ad)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:40.941235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:40.995928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9cbf9b8d2e646eb945deb24d55fa0ad', 'request_id': 'req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:28:41.003048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9cbf9b8d2e646eb945deb24d55fa0ad', 'request_id': 'req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:28:41.009924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9cbf9b8d2e646eb945deb24d55fa0ad', 'request_id': 'req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:28:41.020852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:28:41.025238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] The token cache is being invalidate because project c9cbf9b8d2e646eb945deb24d55fa0ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:28:41.025546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:28:41.026660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fb9ac77c-e564-4d2b-87df-60d6c1769ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:28:41.028526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1164/2295] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:28:40 2022] DELETE /identity/v3/projects/c9cbf9b8d2e646eb945deb24d55fa0ad => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:28:41.035879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92ebc4-09f6-438a-b19f-ef6263d983f8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.036407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92ebc4-09f6-438a-b19f-ef6263d983f8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.036890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92ebc4-09f6-438a-b19f-ef6263d983f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.063655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5a92ebc4-09f6-438a-b19f-ef6263d983f8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:28:41.064853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5a92ebc4-09f6-438a-b19f-ef6263d983f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:41.137065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1132/2296] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:28:41.151069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b116-2de1-44c5-96d9-c30855844195 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.151434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b116-2de1-44c5-96d9-c30855844195 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.151776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0b116-2de1-44c5-96d9-c30855844195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.173454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e0c0b116-2de1-44c5-96d9-c30855844195 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:28:41.174727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0c0b116-2de1-44c5-96d9-c30855844195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:41.183104 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1165/2297] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:28:41.187692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92ebc4-09f6-438a-b19f-ef6263d983f8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:41.202896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92ebc4-09f6-438a-b19f-ef6263d983f8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:41.210935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b42e03-af76-41a3-9490-d1cbcd4330c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:41.212033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b42e03-af76-41a3-9490-d1cbcd4330c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.212503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b42e03-af76-41a3-9490-d1cbcd4330c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.213020 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b42e03-af76-41a3-9490-d1cbcd4330c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.214354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b42e03-af76-41a3-9490-d1cbcd4330c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:28:41.214898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b42e03-af76-41a3-9490-d1cbcd4330c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:41.218422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b42e03-af76-41a3-9490-d1cbcd4330c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:41.235610 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:28:41.245228 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1133/2298] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:28:41.247727 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0b116-2de1-44c5-96d9-c30855844195 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:41.270411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0b116-2de1-44c5-96d9-c30855844195 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:41.277284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2fe88-92ee-4cef-ad3f-d0a99c137781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:41.277976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2fe88-92ee-4cef-ad3f-d0a99c137781 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.278259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2fe88-92ee-4cef-ad3f-d0a99c137781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.278530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef2fe88-92ee-4cef-ad3f-d0a99c137781 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.279277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2fe88-92ee-4cef-ad3f-d0a99c137781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:41.281296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef2fe88-92ee-4cef-ad3f-d0a99c137781 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:41.293534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b42e03-af76-41a3-9490-d1cbcd4330c2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:41.295763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1b42e03-af76-41a3-9490-d1cbcd4330c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:41.325904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1166/2299] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:28:41 2022] POST /identity/v3/projects => generated 389 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:28:41.330215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2fe88-92ee-4cef-ad3f-d0a99c137781 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:41.383270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b42e03-af76-41a3-9490-d1cbcd4330c2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:41.436812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef2fe88-92ee-4cef-ad3f-d0a99c137781 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:41.443652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187f4ac-9283-430a-970e-169c483c91a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:41.444307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f4ac-9283-430a-970e-169c483c91a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.444584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f4ac-9283-430a-970e-169c483c91a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.446486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187f4ac-9283-430a-970e-169c483c91a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.448282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187f4ac-9283-430a-970e-169c483c91a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:41.453146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187f4ac-9283-430a-970e-169c483c91a4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:41.477416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84aaf7-d167-4a8b-a778-1aec38ca793b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:41.478255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84aaf7-d167-4a8b-a778-1aec38ca793b service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.478611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84aaf7-d167-4a8b-a778-1aec38ca793b service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.478967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d84aaf7-d167-4a8b-a778-1aec38ca793b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.481107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d84aaf7-d167-4a8b-a778-1aec38ca793b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:41.491020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1167/2300] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:28:41 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:28:41.496085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187f4ac-9283-430a-970e-169c483c91a4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:41.501709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84aaf7-d167-4a8b-a778-1aec38ca793b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:41.503524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d84aaf7-d167-4a8b-a778-1aec38ca793b service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:41.514787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187f4ac-9283-430a-970e-169c483c91a4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:41.517047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1134/2301] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:28:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:28:41.521791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602c50f-b683-4b5b-99ab-27f3db4bb6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:41.522353 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c50f-b683-4b5b-99ab-27f3db4bb6fb admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.522709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c50f-b683-4b5b-99ab-27f3db4bb6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.523061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c50f-b683-4b5b-99ab-27f3db4bb6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.524039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602c50f-b683-4b5b-99ab-27f3db4bb6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:41.525969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602c50f-b683-4b5b-99ab-27f3db4bb6fb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:41.540951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1168/2302] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:28:41.542516 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84aaf7-d167-4a8b-a778-1aec38ca793b service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:41.557280 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d84aaf7-d167-4a8b-a778-1aec38ca793b service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:41.565354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61b4aa-fae3-4a7f-b7e9-5247a42c5fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:41.566474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61b4aa-fae3-4a7f-b7e9-5247a42c5fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.566968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61b4aa-fae3-4a7f-b7e9-5247a42c5fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.567452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61b4aa-fae3-4a7f-b7e9-5247a42c5fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.568778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61b4aa-fae3-4a7f-b7e9-5247a42c5fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:41.572648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61b4aa-fae3-4a7f-b7e9-5247a42c5fb6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:41.597009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1135/2303] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:28:41.598404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602c50f-b683-4b5b-99ab-27f3db4bb6fb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:41.619119 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602c50f-b683-4b5b-99ab-27f3db4bb6fb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:41.637018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b2a1d-cb43-4a6d-a7b1-605ee0eb21fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:41.640184 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b2a1d-cb43-4a6d-a7b1-605ee0eb21fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.640725 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b2a1d-cb43-4a6d-a7b1-605ee0eb21fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.641257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b2a1d-cb43-4a6d-a7b1-605ee0eb21fd admin admin] PATH_INFO: `/v3/projects/5d14f741553a44758ec58f81e06a3b6c/users/4ee8e2c08c9047849123379bc9d3dd48/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.668248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b2a1d-cb43-4a6d-a7b1-605ee0eb21fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d14f741553a44758ec58f81e06a3b6c, user_id=4ee8e2c08c9047849123379bc9d3dd48, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:41.671555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b2a1d-cb43-4a6d-a7b1-605ee0eb21fd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:41.700165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1169/2304] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:28:41 2022] PUT /identity/v3/projects/5d14f741553a44758ec58f81e06a3b6c/users/4ee8e2c08c9047849123379bc9d3dd48/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:28:41.703622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad2b66-bed6-4acd-b48e-d4a1873bdc58 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.704088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad2b66-bed6-4acd-b48e-d4a1873bdc58 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.704608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad2b66-bed6-4acd-b48e-d4a1873bdc58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.741541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-e4ad2b66-bed6-4acd-b48e-d4a1873bdc58 None None] MFA Rules not processed for user `4ee8e2c08c9047849123379bc9d3dd48`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:28:41.743064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e4ad2b66-bed6-4acd-b48e-d4a1873bdc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:41.835888 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1136/2305] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:28:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:28:41.854849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b2a1d-cb43-4a6d-a7b1-605ee0eb21fd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:28:41.856513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-690b2a1d-cb43-4a6d-a7b1-605ee0eb21fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:28:41.945506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b2a1d-cb43-4a6d-a7b1-605ee0eb21fd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:28:41.951236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c5d21-468c-4868-8804-9dc89bddff1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:28:41.952347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c5d21-468c-4868-8804-9dc89bddff1c service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:28:41.952851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c5d21-468c-4868-8804-9dc89bddff1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:28:41.953387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c5d21-468c-4868-8804-9dc89bddff1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:28:41.970325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c5d21-468c-4868-8804-9dc89bddff1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:28:41.973200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c5d21-468c-4868-8804-9dc89bddff1c service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:28:41.988496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1170/2306] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:28:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:28:51.831620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:28:51.983226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:29:00.752545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0c13e-c883-4385-a39f-eb6d47e5214b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:00.753090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0c13e-c883-4385-a39f-eb6d47e5214b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:00.753090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0c13e-c883-4385-a39f-eb6d47e5214b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:00.773013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-bdf0c13e-c883-4385-a39f-eb6d47e5214b None None] MFA Rules not processed for user `c394476504af4c1d8bea462a5113cebc`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:29:00.774381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdf0c13e-c883-4385-a39f-eb6d47e5214b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:00.838916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1137/2307] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:29:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:29:01.838265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddef44-fc37-4adf-8254-486ca01e6609 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:01.838843 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddef44-fc37-4adf-8254-486ca01e6609 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:01.839296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddef44-fc37-4adf-8254-486ca01e6609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:01.872284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-77ddef44-fc37-4adf-8254-486ca01e6609 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:29:01.873721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77ddef44-fc37-4adf-8254-486ca01e6609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:01.944903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1171/2308] 10.222.0.39 () {58 vars in 967 bytes} [Tue Jul 26 07:29:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:29:01.964827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0c13e-c883-4385-a39f-eb6d47e5214b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:01.966599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-bdf0c13e-c883-4385-a39f-eb6d47e5214b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:02.073777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0c13e-c883-4385-a39f-eb6d47e5214b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:02.083611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5d90b-d84e-462d-af42-8d51c4e2c7a5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:02.085208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5d90b-d84e-462d-af42-8d51c4e2c7a5 service swift] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:02.085835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5d90b-d84e-462d-af42-8d51c4e2c7a5 service swift] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:02.086385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5d90b-d84e-462d-af42-8d51c4e2c7a5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:02.107277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5d90b-d84e-462d-af42-8d51c4e2c7a5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:02.111077 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5d90b-d84e-462d-af42-8d51c4e2c7a5 service swift] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:02.129710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1138/2309] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 07:29:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 07:29:05.150179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ddef44-fc37-4adf-8254-486ca01e6609 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:05.221910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ddef44-fc37-4adf-8254-486ca01e6609 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:05.226659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:05.227319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:05.227600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:05.227869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] PATH_INFO: `/v3/users/4ee8e2c08c9047849123379bc9d3dd48` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:05.229095 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee8e2c08c9047849123379bc9d3dd48)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:05.231133 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:05.293221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee8e2c08c9047849123379bc9d3dd48'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:29:05.293995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee8e2c08c9047849123379bc9d3dd48'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:29:05.307798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee8e2c08c9047849123379bc9d3dd48'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:29:05.316063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee8e2c08c9047849123379bc9d3dd48'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:29:05.326454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee8e2c08c9047849123379bc9d3dd48'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:29:05.333485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1172/2310] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:29:05 2022] DELETE /identity/v3/users/4ee8e2c08c9047849123379bc9d3dd48 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:29:05.338343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eee6f-6e4d-4014-b327-1f2a5992d030 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:05.339015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eee6f-6e4d-4014-b327-1f2a5992d030 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:05.339015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eee6f-6e4d-4014-b327-1f2a5992d030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:05.358891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-cf6eee6f-6e4d-4014-b327-1f2a5992d030 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:29:05.360112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf6eee6f-6e4d-4014-b327-1f2a5992d030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:05.428650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1139/2311] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:29:05.439088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:05.441241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:05.476505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:29:05.512796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a900d-4966-4b28-b97c-6d2dab7499c5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:05.516650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492e8da9-5993-4d5c-b142-8afd72ec1d45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:05.517288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e8da9-5993-4d5c-b142-8afd72ec1d45 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:05.517562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e8da9-5993-4d5c-b142-8afd72ec1d45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:05.517863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492e8da9-5993-4d5c-b142-8afd72ec1d45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:05.527848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e8da9-5993-4d5c-b142-8afd72ec1d45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:05.529392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492e8da9-5993-4d5c-b142-8afd72ec1d45 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:05.540209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1173/2312] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:29:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:29:05.861814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6eee6f-6e4d-4014-b327-1f2a5992d030 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:05.863497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cf6eee6f-6e4d-4014-b327-1f2a5992d030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:05.877372 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6eee6f-6e4d-4014-b327-1f2a5992d030 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:05.882945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:05.883635 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:05.883917 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:05.884194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] PATH_INFO: `/v3/projects/5d14f741553a44758ec58f81e06a3b6c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:05.885493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d14f741553a44758ec58f81e06a3b6c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:05.888365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:05.928926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:29:05.942472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d14f741553a44758ec58f81e06a3b6c', 'request_id': 'req-fd984ed0-6ded-4455-944c-09a7997848c1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:29:05.947872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d14f741553a44758ec58f81e06a3b6c', 'request_id': 'req-fd984ed0-6ded-4455-944c-09a7997848c1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:29:05.956401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d14f741553a44758ec58f81e06a3b6c', 'request_id': 'req-fd984ed0-6ded-4455-944c-09a7997848c1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:29:05.968057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] The token cache is being invalidate because project 5d14f741553a44758ec58f81e06a3b6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:29:05.968338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:29:05.969008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-fd984ed0-6ded-4455-944c-09a7997848c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:29:05.970585 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1140/2313] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:29:05 2022] DELETE /identity/v3/projects/5d14f741553a44758ec58f81e06a3b6c => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:29:05.975724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1512cf8-4812-4d44-9bdf-f3d776421f61 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:05.975986 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1512cf8-4812-4d44-9bdf-f3d776421f61 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:05.976287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1512cf8-4812-4d44-9bdf-f3d776421f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:05.996724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-a1512cf8-4812-4d44-9bdf-f3d776421f61 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:29:05.998017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a1512cf8-4812-4d44-9bdf-f3d776421f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:06.072896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1174/2314] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:29:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:29:06.087804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a4c63-1754-4450-9bfa-24588408941c None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:06.088238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a4c63-1754-4450-9bfa-24588408941c None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:06.088708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a4c63-1754-4450-9bfa-24588408941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:06.115640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d48a4c63-1754-4450-9bfa-24588408941c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:29:06.116850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d48a4c63-1754-4450-9bfa-24588408941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:06.126581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1141/2315] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:29:06.131732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1512cf8-4812-4d44-9bdf-f3d776421f61 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:06.141954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1512cf8-4812-4d44-9bdf-f3d776421f61 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:06.146699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22680d-e36b-4184-9b52-509b41a59cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:06.147355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22680d-e36b-4184-9b52-509b41a59cfa admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:06.147628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22680d-e36b-4184-9b52-509b41a59cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:06.147900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22680d-e36b-4184-9b52-509b41a59cfa admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:06.148598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22680d-e36b-4184-9b52-509b41a59cfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:29:06.148888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22680d-e36b-4184-9b52-509b41a59cfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:06.150547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22680d-e36b-4184-9b52-509b41a59cfa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:06.166453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1175/2316] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:29:06.167814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a4c63-1754-4450-9bfa-24588408941c None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:06.177798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a4c63-1754-4450-9bfa-24588408941c None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:06.183467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f8c5e-7c49-40d0-a067-3231fced9b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:06.184178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f8c5e-7c49-40d0-a067-3231fced9b65 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:06.184469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f8c5e-7c49-40d0-a067-3231fced9b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:06.184748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f8c5e-7c49-40d0-a067-3231fced9b65 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:06.185456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f8c5e-7c49-40d0-a067-3231fced9b65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:06.187423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f8c5e-7c49-40d0-a067-3231fced9b65 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:06.220115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1142/2317] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:29:06 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:29:06.224456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22680d-e36b-4184-9b52-509b41a59cfa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:06.236333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22680d-e36b-4184-9b52-509b41a59cfa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:06.297130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a6a47-b6a1-4557-ae5b-8b85c14b1a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:06.297929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a6a47-b6a1-4557-ae5b-8b85c14b1a24 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:06.298308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a6a47-b6a1-4557-ae5b-8b85c14b1a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:06.298690 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a6a47-b6a1-4557-ae5b-8b85c14b1a24 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:06.299450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a6a47-b6a1-4557-ae5b-8b85c14b1a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:06.312498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53a6a47-b6a1-4557-ae5b-8b85c14b1a24 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:06.327715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1176/2318] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:29:06 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:29:06.332200 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f8c5e-7c49-40d0-a067-3231fced9b65 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:06.343734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f8c5e-7c49-40d0-a067-3231fced9b65 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:06.353105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6d9c2-bb2a-4c3f-8b99-d8be888df188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:06.353763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6d9c2-bb2a-4c3f-8b99-d8be888df188 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:06.354050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6d9c2-bb2a-4c3f-8b99-d8be888df188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:06.354330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f6d9c2-bb2a-4c3f-8b99-d8be888df188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:06.355040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6d9c2-bb2a-4c3f-8b99-d8be888df188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:06.356663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f6d9c2-bb2a-4c3f-8b99-d8be888df188 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:06.370681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1143/2319] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 07:29:06.376525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a6a47-b6a1-4557-ae5b-8b85c14b1a24 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:06.391259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a6a47-b6a1-4557-ae5b-8b85c14b1a24 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:06.396028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc2bf8a-b6d6-46d0-8f2f-d45f6d82524f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:06.396671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2bf8a-b6d6-46d0-8f2f-d45f6d82524f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:06.396948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2bf8a-b6d6-46d0-8f2f-d45f6d82524f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:06.397227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc2bf8a-b6d6-46d0-8f2f-d45f6d82524f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:06.397979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2bf8a-b6d6-46d0-8f2f-d45f6d82524f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:06.399603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc2bf8a-b6d6-46d0-8f2f-d45f6d82524f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:06.411293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1177/2320] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:29:06.415621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6d9c2-bb2a-4c3f-8b99-d8be888df188 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:06.426223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f6d9c2-bb2a-4c3f-8b99-d8be888df188 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:06.431297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c8c32-b81d-498c-90cd-02686bfd0f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:06.432032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c8c32-b81d-498c-90cd-02686bfd0f46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:06.432312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c8c32-b81d-498c-90cd-02686bfd0f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:06.432592 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088c8c32-b81d-498c-90cd-02686bfd0f46 admin admin] PATH_INFO: `/v3/projects/c80c8785a6f7482caf8f72d11937d27f/users/8714dee8553c47f1b12b999f04058b08/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:06.449448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c8c32-b81d-498c-90cd-02686bfd0f46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c80c8785a6f7482caf8f72d11937d27f, user_id=8714dee8553c47f1b12b999f04058b08, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:06.452286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088c8c32-b81d-498c-90cd-02686bfd0f46 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:06.473935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1144/2321] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:29:06 2022] PUT /identity/v3/projects/c80c8785a6f7482caf8f72d11937d27f/users/8714dee8553c47f1b12b999f04058b08/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:29:06.479221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f3187-c5db-4331-8215-85e8dfe043a1 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:06.479864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f3187-c5db-4331-8215-85e8dfe043a1 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:06.480173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f3187-c5db-4331-8215-85e8dfe043a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:06.501629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e98f3187-c5db-4331-8215-85e8dfe043a1 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:29:06.502904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e98f3187-c5db-4331-8215-85e8dfe043a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:06.540655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:29:06.576243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1178/2322] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:29:06.586218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c8c32-b81d-498c-90cd-02686bfd0f46 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:06.588034 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-088c8c32-b81d-498c-90cd-02686bfd0f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:06.660939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088c8c32-b81d-498c-90cd-02686bfd0f46 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:06.668807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fa3f8-9955-459b-b610-990b52b3ebd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:06.670216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa3f8-9955-459b-b610-990b52b3ebd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:06.670707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa3f8-9955-459b-b610-990b52b3ebd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:06.671193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29fa3f8-9955-459b-b610-990b52b3ebd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:06.692169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fa3f8-9955-459b-b610-990b52b3ebd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:06.695076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29fa3f8-9955-459b-b610-990b52b3ebd9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:06.713646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1145/2323] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:29:11.494532 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd5068-9d0f-42c6-9495-22b2877188fb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:11.495071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd5068-9d0f-42c6-9495-22b2877188fb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:11.495272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd5068-9d0f-42c6-9495-22b2877188fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:11.534033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-47bd5068-9d0f-42c6-9495-22b2877188fb None None] MFA Rules not processed for user `8714dee8553c47f1b12b999f04058b08`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:29:11.535263 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-47bd5068-9d0f-42c6-9495-22b2877188fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:11.613272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1179/2324] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:29:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:29:11.618275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fa3f8-9955-459b-b610-990b52b3ebd9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:11.629612 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29fa3f8-9955-459b-b610-990b52b3ebd9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:11.635609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af62834-13f3-4e67-8ac3-507be10de206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:11.635609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af62834-13f3-4e67-8ac3-507be10de206 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:11.635898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af62834-13f3-4e67-8ac3-507be10de206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:11.636132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af62834-13f3-4e67-8ac3-507be10de206 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:11.636823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af62834-13f3-4e67-8ac3-507be10de206 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:11.638873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af62834-13f3-4e67-8ac3-507be10de206 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:11.675266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1146/2325] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:29:11 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:29:11.682973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd5068-9d0f-42c6-9495-22b2877188fb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:11.712261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd5068-9d0f-42c6-9495-22b2877188fb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:11.866169 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecccfc2-1178-4484-8708-53d90e832979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:11.867070 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecccfc2-1178-4484-8708-53d90e832979 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:11.867360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecccfc2-1178-4484-8708-53d90e832979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:11.867641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecccfc2-1178-4484-8708-53d90e832979 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:11.869277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecccfc2-1178-4484-8708-53d90e832979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:11.871533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecccfc2-1178-4484-8708-53d90e832979 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:11.921796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1180/2326] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:29:11 2022] POST /identity/v3/users => generated 293 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:29:11.923649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af62834-13f3-4e67-8ac3-507be10de206 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:11.939710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af62834-13f3-4e67-8ac3-507be10de206 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:11.957541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1210c-2831-46d0-84e0-1f3b65d260ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:11.958264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1210c-2831-46d0-84e0-1f3b65d260ca admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:11.958547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1210c-2831-46d0-84e0-1f3b65d260ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:11.958824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1210c-2831-46d0-84e0-1f3b65d260ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:11.959554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1210c-2831-46d0-84e0-1f3b65d260ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:11.961586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1210c-2831-46d0-84e0-1f3b65d260ca admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:11.984706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1147/2327] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:29:11.985394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecccfc2-1178-4484-8708-53d90e832979 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:11.996754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecccfc2-1178-4484-8708-53d90e832979 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:12.005601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4def3-8290-4251-8ca7-1e0ee76a2b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:12.007278 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4def3-8290-4251-8ca7-1e0ee76a2b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:12.007711 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4def3-8290-4251-8ca7-1e0ee76a2b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:12.008217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4def3-8290-4251-8ca7-1e0ee76a2b43 admin admin] PATH_INFO: `/v3/projects/0162d363d8554c23ace2c1027bc145a0/users/370138b624c349bcb043c7e5f1f2cc2d/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:12.051833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4def3-8290-4251-8ca7-1e0ee76a2b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0162d363d8554c23ace2c1027bc145a0, user_id=370138b624c349bcb043c7e5f1f2cc2d, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:12.056391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4def3-8290-4251-8ca7-1e0ee76a2b43 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:12.080986 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1181/2328] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:29:11 2022] PUT /identity/v3/projects/0162d363d8554c23ace2c1027bc145a0/users/370138b624c349bcb043c7e5f1f2cc2d/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:29:12.085403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1210c-2831-46d0-84e0-1f3b65d260ca admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:12.103461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1210c-2831-46d0-84e0-1f3b65d260ca admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:12.133676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:29:12.201258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21037a-7311-49d0-8263-7229bab3ca2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:12.202544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21037a-7311-49d0-8263-7229bab3ca2a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:12.202945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21037a-7311-49d0-8263-7229bab3ca2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:12.203319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21037a-7311-49d0-8263-7229bab3ca2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:12.204256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21037a-7311-49d0-8263-7229bab3ca2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:12.207366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21037a-7311-49d0-8263-7229bab3ca2a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:12.231016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1148/2329] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:29:12.233229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4def3-8290-4251-8ca7-1e0ee76a2b43 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:12.251631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4def3-8290-4251-8ca7-1e0ee76a2b43 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:12.259304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b105-ae5c-4223-bd45-547b9fe8f3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:12.260172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b105-ae5c-4223-bd45-547b9fe8f3e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:12.260588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b105-ae5c-4223-bd45-547b9fe8f3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:12.260980 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d634b105-ae5c-4223-bd45-547b9fe8f3e7 admin admin] PATH_INFO: `/v3/projects/0162d363d8554c23ace2c1027bc145a0/users/370138b624c349bcb043c7e5f1f2cc2d/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:12.263513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b105-ae5c-4223-bd45-547b9fe8f3e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0162d363d8554c23ace2c1027bc145a0, user_id=370138b624c349bcb043c7e5f1f2cc2d, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:12.266494 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d634b105-ae5c-4223-bd45-547b9fe8f3e7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:12.283776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1182/2330] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:29:12 2022] PUT /identity/v3/projects/0162d363d8554c23ace2c1027bc145a0/users/370138b624c349bcb043c7e5f1f2cc2d/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:29:19.661961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13aead-7099-4d48-865c-b95adab0bcb6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:19.662581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13aead-7099-4d48-865c-b95adab0bcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:19.662581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13aead-7099-4d48-865c-b95adab0bcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:19.694218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9b13aead-7099-4d48-865c-b95adab0bcb6 None None] MFA Rules not processed for user `370138b624c349bcb043c7e5f1f2cc2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:29:19.695352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9b13aead-7099-4d48-865c-b95adab0bcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:19.774759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1149/2331] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:29:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:29:19.789215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b105-ae5c-4223-bd45-547b9fe8f3e7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:29:19.791843 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d634b105-ae5c-4223-bd45-547b9fe8f3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:19.814200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:29:19.888230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d634b105-ae5c-4223-bd45-547b9fe8f3e7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:29:19.894590 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991ede44-4bee-4276-8912-d433aca31fb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:29:19.894927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ede44-4bee-4276-8912-d433aca31fb2 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:19.895089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ede44-4bee-4276-8912-d433aca31fb2 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:19.895895 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ede44-4bee-4276-8912-d433aca31fb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:19.978305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ede44-4bee-4276-8912-d433aca31fb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:29:19.981944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991ede44-4bee-4276-8912-d433aca31fb2 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:29:19.992840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1183/2332] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:29:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:29:21.249772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37521024-7c2d-46d3-96df-aa98f2ce1784 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:21.250336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37521024-7c2d-46d3-96df-aa98f2ce1784 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:21.250673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37521024-7c2d-46d3-96df-aa98f2ce1784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:21.285953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-37521024-7c2d-46d3-96df-aa98f2ce1784 None None] MFA Rules not processed for user `8714dee8553c47f1b12b999f04058b08`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:29:21.287273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-37521024-7c2d-46d3-96df-aa98f2ce1784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:21.294852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1150/2333] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:29:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:29:29.439308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdaad2-4ce1-4ff0-ba97-bafd2e3bf49b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:29:29.440131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdaad2-4ce1-4ff0-ba97-bafd2e3bf49b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:29:29.440291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fdaad2-4ce1-4ff0-ba97-bafd2e3bf49b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:29:29.463947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-19fdaad2-4ce1-4ff0-ba97-bafd2e3bf49b None None] MFA Rules not processed for user `57029844165f4d83892176fe82594685`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:29:29.465199 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19fdaad2-4ce1-4ff0-ba97-bafd2e3bf49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:29:29.534371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1184/2334] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:29:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 07:29:31.282910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=55 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:29:39.522410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=74 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:30:24.351007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37521024-7c2d-46d3-96df-aa98f2ce1784 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:30:24.411659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37521024-7c2d-46d3-96df-aa98f2ce1784 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:30:24.415747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f5577-2bb0-4ba5-a4c2-1260f220a6f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:30:24.416415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f5577-2bb0-4ba5-a4c2-1260f220a6f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:30:24.416695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f5577-2bb0-4ba5-a4c2-1260f220a6f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:30:24.416967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f5577-2bb0-4ba5-a4c2-1260f220a6f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:30:24.419006 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-989f5577-2bb0-4ba5-a4c2-1260f220a6f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:30:24.435462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f5577-2bb0-4ba5-a4c2-1260f220a6f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:30:24.437152 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f5577-2bb0-4ba5-a4c2-1260f220a6f2 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:30:24.462103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1151/2335] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:30:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:30:24.790672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdaad2-4ce1-4ff0-ba97-bafd2e3bf49b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:30:24.792960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19fdaad2-4ce1-4ff0-ba97-bafd2e3bf49b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:30:24.809933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fdaad2-4ce1-4ff0-ba97-bafd2e3bf49b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:30:24.813944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0448dba-65f0-446f-a352-7b0dc51bcd7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:30:24.814601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0448dba-65f0-446f-a352-7b0dc51bcd7b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:30:24.814915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0448dba-65f0-446f-a352-7b0dc51bcd7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:30:24.815155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0448dba-65f0-446f-a352-7b0dc51bcd7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:30:24.817327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0448dba-65f0-446f-a352-7b0dc51bcd7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:30:24.839715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0448dba-65f0-446f-a352-7b0dc51bcd7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:30:24.841540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0448dba-65f0-446f-a352-7b0dc51bcd7b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:30:24.855354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1185/2336] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:30:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:30:30.577039 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f5577-2bb0-4ba5-a4c2-1260f220a6f2 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:30:30.579365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-989f5577-2bb0-4ba5-a4c2-1260f220a6f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:30:30.600164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f5577-2bb0-4ba5-a4c2-1260f220a6f2 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:30:30.609207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e25af8-7b40-4273-9c90-c08f6e67372f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:30:30.609905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e25af8-7b40-4273-9c90-c08f6e67372f service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:30:30.610317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e25af8-7b40-4273-9c90-c08f6e67372f service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:30:30.610600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e25af8-7b40-4273-9c90-c08f6e67372f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:30:30.612857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e0e25af8-7b40-4273-9c90-c08f6e67372f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:30:30.628772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e25af8-7b40-4273-9c90-c08f6e67372f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:30:30.630545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e25af8-7b40-4273-9c90-c08f6e67372f service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:30:30.645773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1152/2337] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:30:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:30:34.850584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:30:36.586578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a688e-9923-43b0-acba-eb854edf7bcd None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:30:36.587728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a688e-9923-43b0-acba-eb854edf7bcd None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:30:36.587728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a688e-9923-43b0-acba-eb854edf7bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:30:36.614596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-b87a688e-9923-43b0-acba-eb854edf7bcd None None] MFA Rules not processed for user `886151888fc84ccf890d609ba62da9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:30:36.616065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b87a688e-9923-43b0-acba-eb854edf7bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:30:36.708839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1186/2338] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:30:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:30:37.233243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e25af8-7b40-4273-9c90-c08f6e67372f service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:30:37.245877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e25af8-7b40-4273-9c90-c08f6e67372f service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:30:37.252359 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37a991-393c-40a4-8480-0b8ead02408f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:30:37.253047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37a991-393c-40a4-8480-0b8ead02408f service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:30:37.253284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37a991-393c-40a4-8480-0b8ead02408f service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:30:37.253560 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f37a991-393c-40a4-8480-0b8ead02408f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:30:37.255616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8f37a991-393c-40a4-8480-0b8ead02408f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:30:37.363577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37a991-393c-40a4-8480-0b8ead02408f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:30:37.365301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f37a991-393c-40a4-8480-0b8ead02408f service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:30:37.379176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1153/2339] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:30:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:30:46.697294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=25 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:30:47.375175 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=60 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:31:14.153261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a688e-9923-43b0-acba-eb854edf7bcd None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:31:14.155432 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b87a688e-9923-43b0-acba-eb854edf7bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:31:14.221902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a688e-9923-43b0-acba-eb854edf7bcd None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:31:14.225692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ef91f-e525-459e-acd3-016a52d117aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:31:14.226352 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ef91f-e525-459e-acd3-016a52d117aa service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:31:14.226629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ef91f-e525-459e-acd3-016a52d117aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:31:14.226899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ef91f-e525-459e-acd3-016a52d117aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:31:14.228917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-623ef91f-e525-459e-acd3-016a52d117aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:31:14.243029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ef91f-e525-459e-acd3-016a52d117aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:31:14.244499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ef91f-e525-459e-acd3-016a52d117aa service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:31:14.254816 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1187/2340] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:31:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:31:22.070606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37a991-393c-40a4-8480-0b8ead02408f service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:31:22.080822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f37a991-393c-40a4-8480-0b8ead02408f service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:31:22.085510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dbc3f-e89c-41e8-9a86-5af998f20987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:31:22.085716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dbc3f-e89c-41e8-9a86-5af998f20987 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:31:22.086031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dbc3f-e89c-41e8-9a86-5af998f20987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:31:22.086304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006dbc3f-e89c-41e8-9a86-5af998f20987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:31:22.088397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-006dbc3f-e89c-41e8-9a86-5af998f20987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:31:22.100616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dbc3f-e89c-41e8-9a86-5af998f20987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:31:22.102212 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006dbc3f-e89c-41e8-9a86-5af998f20987 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:31:22.111502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1154/2341] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:31:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:31:24.250121 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=26 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:31:32.107325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:32:25.110434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ef91f-e525-459e-acd3-016a52d117aa service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:32:25.112028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-623ef91f-e525-459e-acd3-016a52d117aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:32:25.123602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ef91f-e525-459e-acd3-016a52d117aa service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:32:25.127655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a00a61-eeca-4b5f-9421-bbe231c3c1e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:32:25.128286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00a61-eeca-4b5f-9421-bbe231c3c1e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:32:25.128627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00a61-eeca-4b5f-9421-bbe231c3c1e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:32:25.128875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a00a61-eeca-4b5f-9421-bbe231c3c1e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:32:25.130789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-49a00a61-eeca-4b5f-9421-bbe231c3c1e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:32:25.195173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a00a61-eeca-4b5f-9421-bbe231c3c1e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:32:25.197241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a00a61-eeca-4b5f-9421-bbe231c3c1e2 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:32:25.213113 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1188/2342] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:32:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:32:35.206444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=28 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:34:20.705340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dbc3f-e89c-41e8-9a86-5af998f20987 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:34:20.755842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006dbc3f-e89c-41e8-9a86-5af998f20987 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:34:20.761616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ec200-1076-4aa1-ac6b-5bdcf239cb2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:34:20.762759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ec200-1076-4aa1-ac6b-5bdcf239cb2f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:34:20.763282 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ec200-1076-4aa1-ac6b-5bdcf239cb2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:34:20.763837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ec200-1076-4aa1-ac6b-5bdcf239cb2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:34:20.803447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ec200-1076-4aa1-ac6b-5bdcf239cb2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:34:20.804915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ec200-1076-4aa1-ac6b-5bdcf239cb2f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:34:20.815364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1155/2343] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:34:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:34:30.810329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=24 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:34:35.438520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a00a61-eeca-4b5f-9421-bbe231c3c1e2 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:34:35.440012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-49a00a61-eeca-4b5f-9421-bbe231c3c1e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:34:35.588434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a00a61-eeca-4b5f-9421-bbe231c3c1e2 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:34:35.594503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c9e86-4a77-4ba5-b119-db41d7930735 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:34:35.595166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c9e86-4a77-4ba5-b119-db41d7930735 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:34:35.595426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c9e86-4a77-4ba5-b119-db41d7930735 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:34:35.595700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8c9e86-4a77-4ba5-b119-db41d7930735 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:34:35.597620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c8c9e86-4a77-4ba5-b119-db41d7930735 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:34:35.612073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c9e86-4a77-4ba5-b119-db41d7930735 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:34:35.612073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8c9e86-4a77-4ba5-b119-db41d7930735 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:34:35.622762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1189/2344] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:34:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:34:45.618242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=59 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:35:22.886559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ec200-1076-4aa1-ac6b-5bdcf239cb2f service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:35:22.907515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ec200-1076-4aa1-ac6b-5bdcf239cb2f service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:35:22.911772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fa205-954f-402e-8271-97353925dc3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:35:22.912483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fa205-954f-402e-8271-97353925dc3f service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:35:22.912769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fa205-954f-402e-8271-97353925dc3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:35:22.913022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fa205-954f-402e-8271-97353925dc3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:35:22.990429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fa205-954f-402e-8271-97353925dc3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:35:22.991883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fa205-954f-402e-8271-97353925dc3f service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:35:23.012189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1156/2345] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:35:26.355097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c9e86-4a77-4ba5-b119-db41d7930735 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:35:26.356744 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8c8c9e86-4a77-4ba5-b119-db41d7930735 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:35:26.371022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8c9e86-4a77-4ba5-b119-db41d7930735 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:35:26.376748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb3da7-4514-4e56-a56f-30c791b25ae9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:35:26.377480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb3da7-4514-4e56-a56f-30c791b25ae9 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:35:26.378027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb3da7-4514-4e56-a56f-30c791b25ae9 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:35:26.378440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb3da7-4514-4e56-a56f-30c791b25ae9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:35:26.381362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-45cb3da7-4514-4e56-a56f-30c791b25ae9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:35:26.397618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb3da7-4514-4e56-a56f-30c791b25ae9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:35:26.401183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb3da7-4514-4e56-a56f-30c791b25ae9 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:35:26.417890 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1190/2346] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:35:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:35:30.136955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fa205-954f-402e-8271-97353925dc3f service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:35:30.172040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fa205-954f-402e-8271-97353925dc3f service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:35:30.180070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76da68f-3ee4-4c01-a436-2f1d0ee624e6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:35:30.181353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76da68f-3ee4-4c01-a436-2f1d0ee624e6 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:35:30.186082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76da68f-3ee4-4c01-a436-2f1d0ee624e6 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:35:30.186082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76da68f-3ee4-4c01-a436-2f1d0ee624e6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:35:30.190784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e76da68f-3ee4-4c01-a436-2f1d0ee624e6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:35:30.211028 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76da68f-3ee4-4c01-a436-2f1d0ee624e6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:35:30.216404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76da68f-3ee4-4c01-a436-2f1d0ee624e6 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:35:30.220612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb3da7-4514-4e56-a56f-30c791b25ae9 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:35:30.244123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1157/2347] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:35:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:35:30.246602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb3da7-4514-4e56-a56f-30c791b25ae9 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:35:30.256842 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:35:30.257505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:35:30.258224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:35:30.258665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] PATH_INFO: `/v3/users/8714dee8553c47f1b12b999f04058b08` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:35:30.260540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8714dee8553c47f1b12b999f04058b08)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:35:30.262910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:35:30.318097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8714dee8553c47f1b12b999f04058b08'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.319193 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8714dee8553c47f1b12b999f04058b08'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.332962 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8714dee8553c47f1b12b999f04058b08'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.341012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8714dee8553c47f1b12b999f04058b08'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.353567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8714dee8553c47f1b12b999f04058b08'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.360951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1191/2348] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:35:30 2022] DELETE /identity/v3/users/8714dee8553c47f1b12b999f04058b08 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:35:30.570814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76da68f-3ee4-4c01-a436-2f1d0ee624e6 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:35:30.590328 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76da68f-3ee4-4c01-a436-2f1d0ee624e6 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:35:30.677534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:35:30.678560 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:35:30.678954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:35:30.679318 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] PATH_INFO: `/v3/projects/c80c8785a6f7482caf8f72d11937d27f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:35:30.680965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c80c8785a6f7482caf8f72d11937d27f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:35:30.683864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:35:30.745540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c80c8785a6f7482caf8f72d11937d27f', 'request_id': 'req-561d6786-5fd4-4be4-9698-fd2131a921a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.752231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:35:30.753828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c80c8785a6f7482caf8f72d11937d27f', 'request_id': 'req-561d6786-5fd4-4be4-9698-fd2131a921a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.766682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c80c8785a6f7482caf8f72d11937d27f', 'request_id': 'req-561d6786-5fd4-4be4-9698-fd2131a921a0'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.794237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] The token cache is being invalidate because project c80c8785a6f7482caf8f72d11937d27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:35:30.794809 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.795688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.797289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1158/2349] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:35:30 2022] DELETE /identity/v3/projects/c80c8785a6f7482caf8f72d11937d27f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:35:30.802078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:35:30.803731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:35:30.818038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab60fe4-65ec-4af5-b279-00858737a422 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:35:30.884589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:35:30.885246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:35:30.885526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:35:30.885824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] PATH_INFO: `/v3/users/370138b624c349bcb043c7e5f1f2cc2d` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:35:30.887017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=370138b624c349bcb043c7e5f1f2cc2d)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:35:30.888960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:35:30.944908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370138b624c349bcb043c7e5f1f2cc2d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.945652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370138b624c349bcb043c7e5f1f2cc2d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.949970 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:35:30.951699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:35:30.960841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:35:30.962939 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370138b624c349bcb043c7e5f1f2cc2d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.972084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370138b624c349bcb043c7e5f1f2cc2d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.981359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370138b624c349bcb043c7e5f1f2cc2d'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:30.992874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1192/2350] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:35:30 2022] DELETE /identity/v3/users/370138b624c349bcb043c7e5f1f2cc2d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:35:31.035584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d6786-5fd4-4be4-9698-fd2131a921a0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:35:31.093584 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8b3fc-952b-4099-b16e-3dc7a11a4962 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:35:31.094502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8b3fc-952b-4099-b16e-3dc7a11a4962 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:35:31.096062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8b3fc-952b-4099-b16e-3dc7a11a4962 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:35:31.096615 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd8b3fc-952b-4099-b16e-3dc7a11a4962 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:35:31.099766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9dd8b3fc-952b-4099-b16e-3dc7a11a4962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:35:31.199719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:35:31.201160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:35:31.223989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8b3fc-952b-4099-b16e-3dc7a11a4962 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:35:31.225470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd8b3fc-952b-4099-b16e-3dc7a11a4962 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:35:31.246478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1159/2351] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:35:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:35:31.300533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b78a368-6f5c-4ed4-9472-4b6c5ce78d13 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:35:31.310395 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:35:31.318949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:35:31.318949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:35:31.318949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] PATH_INFO: `/v3/projects/0162d363d8554c23ace2c1027bc145a0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:35:31.329976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0162d363d8554c23ace2c1027bc145a0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:35:31.329976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:35:31.382940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0162d363d8554c23ace2c1027bc145a0', 'request_id': 'req-9b69deac-c051-4533-8a23-35bbf1205ad0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:31.390206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0162d363d8554c23ace2c1027bc145a0', 'request_id': 'req-9b69deac-c051-4533-8a23-35bbf1205ad0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:31.397634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0162d363d8554c23ace2c1027bc145a0', 'request_id': 'req-9b69deac-c051-4533-8a23-35bbf1205ad0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:31.411103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] The token cache is being invalidate because project 0162d363d8554c23ace2c1027bc145a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:35:31.411613 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:31.412048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:35:31.413511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1193/2352] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:35:31 2022] DELETE /identity/v3/projects/0162d363d8554c23ace2c1027bc145a0 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:35:36.631003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8b3fc-952b-4099-b16e-3dc7a11a4962 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:35:36.632547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9dd8b3fc-952b-4099-b16e-3dc7a11a4962 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:35:36.637395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:35:36.762355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd8b3fc-952b-4099-b16e-3dc7a11a4962 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:35:36.842969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd84062-b03e-4568-b237-9171c73ffeb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:35:36.843672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd84062-b03e-4568-b237-9171c73ffeb1 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:35:36.843954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd84062-b03e-4568-b237-9171c73ffeb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:35:36.844285 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd84062-b03e-4568-b237-9171c73ffeb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:35:36.846456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4dd84062-b03e-4568-b237-9171c73ffeb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:35:36.938919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:35:36.943398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd84062-b03e-4568-b237-9171c73ffeb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:35:36.945243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd84062-b03e-4568-b237-9171c73ffeb1 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:35:36.966568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1160/2353] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:35:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:35:41.407841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=9,keystone:SELECT=58,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:35:46.963295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=7 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:36:13.773876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:36:13.775640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:36:13.793696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69deac-c051-4533-8a23-35bbf1205ad0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:36:13.802472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54403b8-ca20-4436-8421-b62ee1a06b12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:36:13.803389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54403b8-ca20-4436-8421-b62ee1a06b12 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:36:13.803784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54403b8-ca20-4436-8421-b62ee1a06b12 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:36:13.804129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54403b8-ca20-4436-8421-b62ee1a06b12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:36:13.807407 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d54403b8-ca20-4436-8421-b62ee1a06b12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:36:13.904632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54403b8-ca20-4436-8421-b62ee1a06b12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:36:13.906259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54403b8-ca20-4436-8421-b62ee1a06b12 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:36:13.925998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1194/2354] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:36:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:36:23.922984 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=41 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:36:37.103820 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd84062-b03e-4568-b237-9171c73ffeb1 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:36:37.105484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4dd84062-b03e-4568-b237-9171c73ffeb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:36:37.181413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd84062-b03e-4568-b237-9171c73ffeb1 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:36:37.185442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94664e7-e825-4ba6-b610-594a80b1e9a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:36:37.186103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94664e7-e825-4ba6-b610-594a80b1e9a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:36:37.186384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94664e7-e825-4ba6-b610-594a80b1e9a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:36:37.186655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94664e7-e825-4ba6-b610-594a80b1e9a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:36:37.188425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c94664e7-e825-4ba6-b610-594a80b1e9a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:36:37.198981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94664e7-e825-4ba6-b610-594a80b1e9a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:36:37.200395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94664e7-e825-4ba6-b610-594a80b1e9a3 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:36:37.209018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1161/2355] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:36:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:36:47.205136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=31 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:37:11.069769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54403b8-ca20-4436-8421-b62ee1a06b12 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:37:11.080216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54403b8-ca20-4436-8421-b62ee1a06b12 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:37:11.084428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b3579-696b-4bc7-99e7-9f6bc154163f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:37:11.085043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b3579-696b-4bc7-99e7-9f6bc154163f service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:37:11.085319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b3579-696b-4bc7-99e7-9f6bc154163f service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:37:11.085603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b3579-696b-4bc7-99e7-9f6bc154163f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:37:11.087449 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe7b3579-696b-4bc7-99e7-9f6bc154163f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:11.102258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b3579-696b-4bc7-99e7-9f6bc154163f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:37:11.103805 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7b3579-696b-4bc7-99e7-9f6bc154163f service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:37:11.116800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1195/2356] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:37:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:37:12.354570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94664e7-e825-4ba6-b610-594a80b1e9a3 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:37:12.383142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94664e7-e825-4ba6-b610-594a80b1e9a3 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:37:12.390764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4abf6-4c2d-4f91-88b0-516796f74269 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:37:12.398832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4abf6-4c2d-4f91-88b0-516796f74269 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:37:12.399216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4abf6-4c2d-4f91-88b0-516796f74269 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:37:12.399501 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4abf6-4c2d-4f91-88b0-516796f74269 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:37:12.401782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-79b4abf6-4c2d-4f91-88b0-516796f74269 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:12.416407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4abf6-4c2d-4f91-88b0-516796f74269 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:37:12.418660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b4abf6-4c2d-4f91-88b0-516796f74269 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:37:12.443708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1162/2357] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:37:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:37:16.060523 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cd6e9-eb68-4934-90cb-e1bd34de9d33 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:37:16.061117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cd6e9-eb68-4934-90cb-e1bd34de9d33 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:37:16.061117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1cd6e9-eb68-4934-90cb-e1bd34de9d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:37:16.089768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0b1cd6e9-eb68-4934-90cb-e1bd34de9d33 None None] MFA Rules not processed for user `57029844165f4d83892176fe82594685`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:37:16.090993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0b1cd6e9-eb68-4934-90cb-e1bd34de9d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:16.147066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1196/2358] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:37:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:37:16.187458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4abf6-4c2d-4f91-88b0-516796f74269 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:37:16.188966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-79b4abf6-4c2d-4f91-88b0-516796f74269 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:16.268705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4abf6-4c2d-4f91-88b0-516796f74269 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:37:16.272885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f36be-6b03-4e31-8372-bbf0d2333609 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c6943686c843e588df14f7c6dffb8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:37:16.273515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f36be-6b03-4e31-8372-bbf0d2333609 service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:37:16.273867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f36be-6b03-4e31-8372-bbf0d2333609 service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:37:16.274141 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f36be-6b03-4e31-8372-bbf0d2333609 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:37:16.276203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7f3f36be-6b03-4e31-8372-bbf0d2333609 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:16.290825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f36be-6b03-4e31-8372-bbf0d2333609 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:37:16.292275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f36be-6b03-4e31-8372-bbf0d2333609 service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:37:16.305108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1163/2359] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:37:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:37:16.598112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cd6e9-eb68-4934-90cb-e1bd34de9d33 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:37:16.600123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0b1cd6e9-eb68-4934-90cb-e1bd34de9d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:16.614177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1cd6e9-eb68-4934-90cb-e1bd34de9d33 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:37:16.619209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c39c50-348e-46e7-8adb-bc4e29637dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:37:16.619836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c39c50-348e-46e7-8adb-bc4e29637dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:37:16.620122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c39c50-348e-46e7-8adb-bc4e29637dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:37:16.620422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c39c50-348e-46e7-8adb-bc4e29637dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:37:16.623361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b3c39c50-348e-46e7-8adb-bc4e29637dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:16.634027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c39c50-348e-46e7-8adb-bc4e29637dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:37:16.635477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c39c50-348e-46e7-8adb-bc4e29637dd1 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:37:16.643971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1197/2360] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:37:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:37:18.006499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f36be-6b03-4e31-8372-bbf0d2333609 service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:37:18.008013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7f3f36be-6b03-4e31-8372-bbf0d2333609 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:18.081606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f36be-6b03-4e31-8372-bbf0d2333609 service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:37:18.086837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564a45f-0634-4e4b-b585-aa022914a160 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:37:18.087812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564a45f-0634-4e4b-b585-aa022914a160 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:37:18.088090 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564a45f-0634-4e4b-b585-aa022914a160 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:37:18.088369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f564a45f-0634-4e4b-b585-aa022914a160 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:37:18.090220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f564a45f-0634-4e4b-b585-aa022914a160 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:18.102084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564a45f-0634-4e4b-b585-aa022914a160 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:37:18.103808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f564a45f-0634-4e4b-b585-aa022914a160 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:37:18.114529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1164/2361] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:37:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:37:25.523250 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c39c50-348e-46e7-8adb-bc4e29637dd1 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:37:25.525159 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b3c39c50-348e-46e7-8adb-bc4e29637dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:25.583438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c39c50-348e-46e7-8adb-bc4e29637dd1 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:37:25.587837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef00ec-fcb0-4c7c-b47c-15a096426167 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:37:25.588463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef00ec-fcb0-4c7c-b47c-15a096426167 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:37:25.588749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef00ec-fcb0-4c7c-b47c-15a096426167 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:37:25.588998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef00ec-fcb0-4c7c-b47c-15a096426167 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:37:25.590944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-36ef00ec-fcb0-4c7c-b47c-15a096426167 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:25.674751 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef00ec-fcb0-4c7c-b47c-15a096426167 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:37:25.676178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef00ec-fcb0-4c7c-b47c-15a096426167 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:37:25.690566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:37:25.696079 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1198/2362] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:37:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:37:26.779357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564a45f-0634-4e4b-b585-aa022914a160 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:37:26.799079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f564a45f-0634-4e4b-b585-aa022914a160 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:37:26.815316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a346632-73d3-4a9a-a66c-6648aba37457 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:37:26.816006 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a346632-73d3-4a9a-a66c-6648aba37457 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:37:26.816291 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a346632-73d3-4a9a-a66c-6648aba37457 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:37:26.816936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a346632-73d3-4a9a-a66c-6648aba37457 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:37:26.819690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9a346632-73d3-4a9a-a66c-6648aba37457 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:37:26.836456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a346632-73d3-4a9a-a66c-6648aba37457 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:37:26.839256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a346632-73d3-4a9a-a66c-6648aba37457 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:37:26.851762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1165/2363] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:37:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:37:35.695209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:37:36.846611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=79 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:39:39.372608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef00ec-fcb0-4c7c-b47c-15a096426167 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:39:39.385686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef00ec-fcb0-4c7c-b47c-15a096426167 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:39:39.391842 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b322ba-9392-4926-87e0-97e8631ca041 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:39:39.392491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b322ba-9392-4926-87e0-97e8631ca041 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:39:39.392773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b322ba-9392-4926-87e0-97e8631ca041 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:39:39.393046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b322ba-9392-4926-87e0-97e8631ca041 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:39:39.394992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a8b322ba-9392-4926-87e0-97e8631ca041 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:39:39.408434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b322ba-9392-4926-87e0-97e8631ca041 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:39:39.410115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b322ba-9392-4926-87e0-97e8631ca041 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:39:39.422184 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1199/2364] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:39:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:39:49.417161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:40:32.227562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a346632-73d3-4a9a-a66c-6648aba37457 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:40:32.239904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a346632-73d3-4a9a-a66c-6648aba37457 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:40:32.246676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064650e-82b4-4355-bc31-9c53ffea7a87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:40:32.247547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064650e-82b4-4355-bc31-9c53ffea7a87 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:40:32.247838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064650e-82b4-4355-bc31-9c53ffea7a87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:40:32.248106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064650e-82b4-4355-bc31-9c53ffea7a87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:40:32.250997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d064650e-82b4-4355-bc31-9c53ffea7a87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:40:32.266769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064650e-82b4-4355-bc31-9c53ffea7a87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:40:32.268199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d064650e-82b4-4355-bc31-9c53ffea7a87 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:40:32.278725 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1166/2365] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:40:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:40:37.147651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b322ba-9392-4926-87e0-97e8631ca041 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:40:37.157471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b322ba-9392-4926-87e0-97e8631ca041 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:40:37.161879 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efbb0f-9b77-49da-84d6-b15997931fab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:40:37.162469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efbb0f-9b77-49da-84d6-b15997931fab service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:40:37.162766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efbb0f-9b77-49da-84d6-b15997931fab service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:40:37.163031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efbb0f-9b77-49da-84d6-b15997931fab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:40:37.171766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efbb0f-9b77-49da-84d6-b15997931fab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:40:37.173179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efbb0f-9b77-49da-84d6-b15997931fab service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:40:37.182303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1200/2366] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:40:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:40:42.273443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:40:47.177965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:41:13.689111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064650e-82b4-4355-bc31-9c53ffea7a87 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:41:13.700647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064650e-82b4-4355-bc31-9c53ffea7a87 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:41:13.705515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286d544-ed5e-4d57-bdd6-ef0d88038a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:41:13.706174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286d544-ed5e-4d57-bdd6-ef0d88038a7d service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:41:13.706450 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286d544-ed5e-4d57-bdd6-ef0d88038a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:41:13.706715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4286d544-ed5e-4d57-bdd6-ef0d88038a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:41:13.716870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286d544-ed5e-4d57-bdd6-ef0d88038a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:41:13.718346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4286d544-ed5e-4d57-bdd6-ef0d88038a7d service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:41:13.728226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1167/2367] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:41:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:41:23.723681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:42:11.375118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efbb0f-9b77-49da-84d6-b15997931fab service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:42:11.386836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efbb0f-9b77-49da-84d6-b15997931fab service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:42:11.392019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06214d85-ddcb-4098-95c8-03d3d3907472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:42:11.392635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06214d85-ddcb-4098-95c8-03d3d3907472 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:42:11.392919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06214d85-ddcb-4098-95c8-03d3d3907472 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:42:11.393186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06214d85-ddcb-4098-95c8-03d3d3907472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:42:11.405783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06214d85-ddcb-4098-95c8-03d3d3907472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:42:11.407255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06214d85-ddcb-4098-95c8-03d3d3907472 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:42:11.419439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1201/2368] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:42:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:42:16.686668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286d544-ed5e-4d57-bdd6-ef0d88038a7d service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:42:16.701029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4286d544-ed5e-4d57-bdd6-ef0d88038a7d service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:42:16.705483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88606115-51dc-41f2-a500-a708de7c0c94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:42:16.706505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88606115-51dc-41f2-a500-a708de7c0c94 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:42:16.706783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88606115-51dc-41f2-a500-a708de7c0c94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:42:16.707050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88606115-51dc-41f2-a500-a708de7c0c94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:42:16.719160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88606115-51dc-41f2-a500-a708de7c0c94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:42:16.720588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88606115-51dc-41f2-a500-a708de7c0c94 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:42:16.731874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1168/2369] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:42:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:42:21.413651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:42:26.145440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06214d85-ddcb-4098-95c8-03d3d3907472 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:42:26.157183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06214d85-ddcb-4098-95c8-03d3d3907472 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:42:26.164314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a51fdf-ccb2-4d42-a853-700196caa247 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:42:26.164926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a51fdf-ccb2-4d42-a853-700196caa247 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:42:26.165210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a51fdf-ccb2-4d42-a853-700196caa247 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:42:26.165478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a51fdf-ccb2-4d42-a853-700196caa247 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:42:26.174329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a51fdf-ccb2-4d42-a853-700196caa247 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:42:26.175932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a51fdf-ccb2-4d42-a853-700196caa247 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:42:26.187876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1202/2370] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:42:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:42:26.726980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:42:36.180979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:42:44.100640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88606115-51dc-41f2-a500-a708de7c0c94 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:42:44.110072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88606115-51dc-41f2-a500-a708de7c0c94 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:42:44.114213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bfe44-9c3f-4033-abc2-ab2d145c57b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:42:44.114875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bfe44-9c3f-4033-abc2-ab2d145c57b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:42:44.115160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bfe44-9c3f-4033-abc2-ab2d145c57b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:42:44.115433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bfe44-9c3f-4033-abc2-ab2d145c57b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:42:44.123413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bfe44-9c3f-4033-abc2-ab2d145c57b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:42:44.124840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bfe44-9c3f-4033-abc2-ab2d145c57b9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:42:44.133425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1169/2371] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:42:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:42:54.129813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:44:22.889245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a51fdf-ccb2-4d42-a853-700196caa247 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:44:22.890789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-86a51fdf-ccb2-4d42-a853-700196caa247 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:44:22.915999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a51fdf-ccb2-4d42-a853-700196caa247 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:44:22.922095 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d833a-b72e-41de-8467-e3b1c13b6246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:44:22.922950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d833a-b72e-41de-8467-e3b1c13b6246 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:44:22.923390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d833a-b72e-41de-8467-e3b1c13b6246 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:44:22.923848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d833a-b72e-41de-8467-e3b1c13b6246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:44:22.935658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d833a-b72e-41de-8467-e3b1c13b6246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:44:22.937693 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d833a-b72e-41de-8467-e3b1c13b6246 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:44:22.950794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1203/2372] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:44:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:44:32.945183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:44:48.458378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bfe44-9c3f-4033-abc2-ab2d145c57b9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:44:48.515652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bfe44-9c3f-4033-abc2-ab2d145c57b9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:44:48.521154 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8368cd-72b7-47a5-a5d6-aabee127736d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:44:48.521816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8368cd-72b7-47a5-a5d6-aabee127736d service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:44:48.522109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8368cd-72b7-47a5-a5d6-aabee127736d service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:44:48.522366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8368cd-72b7-47a5-a5d6-aabee127736d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:44:48.531635 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8368cd-72b7-47a5-a5d6-aabee127736d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:44:48.533229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8368cd-72b7-47a5-a5d6-aabee127736d service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:44:48.545029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1170/2373] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:44:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:44:58.539414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=24 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:45:37.138362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d833a-b72e-41de-8467-e3b1c13b6246 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:45:37.258275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d833a-b72e-41de-8467-e3b1c13b6246 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:45:37.264535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b5492-aed4-47b5-8084-25d8f428768a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:45:37.265084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b5492-aed4-47b5-8084-25d8f428768a service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:45:37.265362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b5492-aed4-47b5-8084-25d8f428768a service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:45:37.265636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6b5492-aed4-47b5-8084-25d8f428768a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:45:37.267880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2a6b5492-aed4-47b5-8084-25d8f428768a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:45:37.363541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b5492-aed4-47b5-8084-25d8f428768a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:45:37.365167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6b5492-aed4-47b5-8084-25d8f428768a service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:45:37.383846 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1204/2374] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:45:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:45:42.889322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8368cd-72b7-47a5-a5d6-aabee127736d service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:45:42.910883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8368cd-72b7-47a5-a5d6-aabee127736d service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:45:42.915604 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dbb1d-8768-4db3-9570-85e49a277306 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:45:42.916231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dbb1d-8768-4db3-9570-85e49a277306 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:45:42.916540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dbb1d-8768-4db3-9570-85e49a277306 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:45:42.916788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6dbb1d-8768-4db3-9570-85e49a277306 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:45:42.927002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6dbb1d-8768-4db3-9570-85e49a277306 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:45:42.928432 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6dbb1d-8768-4db3-9570-85e49a277306 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:45:42.957311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1171/2375] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:45:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:45:47.381290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:45:52.954458 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=17 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:46:13.627394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b5492-aed4-47b5-8084-25d8f428768a service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:46:13.651548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6b5492-aed4-47b5-8084-25d8f428768a service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:46:13.657046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d6e5c-3c52-4227-a68f-8bd8be45b328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:46:13.657685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d6e5c-3c52-4227-a68f-8bd8be45b328 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:46:13.658004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d6e5c-3c52-4227-a68f-8bd8be45b328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:46:13.658277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d6e5c-3c52-4227-a68f-8bd8be45b328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:46:13.668595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d6e5c-3c52-4227-a68f-8bd8be45b328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:46:13.670089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d6e5c-3c52-4227-a68f-8bd8be45b328 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:46:13.679581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1205/2376] 10.222.0.39 () {60 vars in 1396 bytes} [Tue Jul 26 07:46:13 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:46:23.675348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:47:10.862900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dbb1d-8768-4db3-9570-85e49a277306 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:47:10.864540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9d6dbb1d-8768-4db3-9570-85e49a277306 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:10.932511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6dbb1d-8768-4db3-9570-85e49a277306 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:47:10.936909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2486-47a7-4afa-93fa-e90f4d55bd89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:47:10.937525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2486-47a7-4afa-93fa-e90f4d55bd89 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:47:10.937837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2486-47a7-4afa-93fa-e90f4d55bd89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:47:10.938108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e2486-47a7-4afa-93fa-e90f4d55bd89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:47:10.948422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e2486-47a7-4afa-93fa-e90f4d55bd89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:47:10.949779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e2486-47a7-4afa-93fa-e90f4d55bd89 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:47:10.962088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1172/2377] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:47:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:47:16.872371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d6e5c-3c52-4227-a68f-8bd8be45b328 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:47:16.874358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f42d6e5c-3c52-4227-a68f-8bd8be45b328 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:16.933013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d6e5c-3c52-4227-a68f-8bd8be45b328 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:47:16.937114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451a434-c2e1-4482-9ddf-02da64268edb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:47:16.937653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451a434-c2e1-4482-9ddf-02da64268edb service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:47:16.937959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451a434-c2e1-4482-9ddf-02da64268edb service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:47:16.938210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451a434-c2e1-4482-9ddf-02da64268edb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:47:16.940236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b451a434-c2e1-4482-9ddf-02da64268edb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:16.968063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451a434-c2e1-4482-9ddf-02da64268edb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:47:16.969528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451a434-c2e1-4482-9ddf-02da64268edb service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:47:16.981538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1206/2378] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:47:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:47:20.956831 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=31 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:47:26.975872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=31 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:47:43.155073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2486-47a7-4afa-93fa-e90f4d55bd89 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:47:43.156671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-307e2486-47a7-4afa-93fa-e90f4d55bd89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:43.230392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e2486-47a7-4afa-93fa-e90f4d55bd89 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:47:43.234545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a512661-6e8b-410e-96a7-face569c86d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:47:43.235176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512661-6e8b-410e-96a7-face569c86d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:47:43.235459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512661-6e8b-410e-96a7-face569c86d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:47:43.235730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a512661-6e8b-410e-96a7-face569c86d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:47:43.237674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6a512661-6e8b-410e-96a7-face569c86d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:43.324471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a512661-6e8b-410e-96a7-face569c86d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:47:43.325951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a512661-6e8b-410e-96a7-face569c86d5 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:47:43.346576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1173/2379] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:47:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:47:44.010448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd92bb6-9896-4a43-9dd5-559084d71dd3 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:47:44.011078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd92bb6-9896-4a43-9dd5-559084d71dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:47:44.011078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd92bb6-9896-4a43-9dd5-559084d71dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:47:44.041565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dfd92bb6-9896-4a43-9dd5-559084d71dd3 None None] MFA Rules not processed for user `08c4ec6a688c4608aa75aaf046933eec`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:47:44.042811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dfd92bb6-9896-4a43-9dd5-559084d71dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:44.050549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1207/2380] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:47:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:47:44.365168 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a512661-6e8b-410e-96a7-face569c86d5 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:47:44.376543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a512661-6e8b-410e-96a7-face569c86d5 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:47:44.380771 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2d73f-d563-45ee-80c5-f2503937bafc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:47:44.381403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2d73f-d563-45ee-80c5-f2503937bafc service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:47:44.381681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2d73f-d563-45ee-80c5-f2503937bafc service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:47:44.381993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2d73f-d563-45ee-80c5-f2503937bafc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:47:44.383857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b9a2d73f-d563-45ee-80c5-f2503937bafc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:44.394886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2d73f-d563-45ee-80c5-f2503937bafc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:47:44.396324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2d73f-d563-45ee-80c5-f2503937bafc service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:47:44.405089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1174/2381] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:47:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:47:44.729190 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd92bb6-9896-4a43-9dd5-559084d71dd3 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:47:44.730662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dfd92bb6-9896-4a43-9dd5-559084d71dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:44.818719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd92bb6-9896-4a43-9dd5-559084d71dd3 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:47:44.824269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3baede-ed9e-4472-a722-3e301510f473 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:47:44.824891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3baede-ed9e-4472-a722-3e301510f473 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:47:44.825172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3baede-ed9e-4472-a722-3e301510f473 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:47:44.825446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3baede-ed9e-4472-a722-3e301510f473 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:47:44.827946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f3baede-ed9e-4472-a722-3e301510f473 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:44.839378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3baede-ed9e-4472-a722-3e301510f473 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:47:44.841001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3baede-ed9e-4472-a722-3e301510f473 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:47:44.850781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1208/2382] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:47:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:47:46.558847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2d73f-d563-45ee-80c5-f2503937bafc service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:47:46.567963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2d73f-d563-45ee-80c5-f2503937bafc service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:47:46.572001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec41a2-df93-4bb1-a8ee-9cae7106d835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:47:46.572661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec41a2-df93-4bb1-a8ee-9cae7106d835 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:47:46.572943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec41a2-df93-4bb1-a8ee-9cae7106d835 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:47:46.573216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ec41a2-df93-4bb1-a8ee-9cae7106d835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:47:46.575172 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-28ec41a2-df93-4bb1-a8ee-9cae7106d835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:46.586555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec41a2-df93-4bb1-a8ee-9cae7106d835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:47:46.588122 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ec41a2-df93-4bb1-a8ee-9cae7106d835 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:47:46.597123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1175/2383] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:47:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:47:51.816600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3baede-ed9e-4472-a722-3e301510f473 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:47:51.830460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3baede-ed9e-4472-a722-3e301510f473 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:47:51.842068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3b853-a4e3-4d9d-834a-8fff5116bd0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:47:51.842721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3b853-a4e3-4d9d-834a-8fff5116bd0b service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:47:51.843002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3b853-a4e3-4d9d-834a-8fff5116bd0b service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:47:51.843276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c3b853-a4e3-4d9d-834a-8fff5116bd0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:47:51.845333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77c3b853-a4e3-4d9d-834a-8fff5116bd0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:47:51.858450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3b853-a4e3-4d9d-834a-8fff5116bd0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:47:51.860241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c3b853-a4e3-4d9d-834a-8fff5116bd0b service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:47:51.870961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1209/2384] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:47:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:47:56.593257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=78 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:48:01.866497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=49 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:49:33.956050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec41a2-df93-4bb1-a8ee-9cae7106d835 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:49:33.972694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ec41a2-df93-4bb1-a8ee-9cae7106d835 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:49:33.980240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e960886-7d58-4853-8063-aa5371c42775 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:49:33.980995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e960886-7d58-4853-8063-aa5371c42775 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:49:33.981289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e960886-7d58-4853-8063-aa5371c42775 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:49:33.981549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e960886-7d58-4853-8063-aa5371c42775 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:49:33.986697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e960886-7d58-4853-8063-aa5371c42775 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:49:34.002009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e960886-7d58-4853-8063-aa5371c42775 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:49:34.003816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e960886-7d58-4853-8063-aa5371c42775 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:49:34.015786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1176/2385] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:49:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:49:44.012107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:49:55.356137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3b853-a4e3-4d9d-834a-8fff5116bd0b service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:49:55.357909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-77c3b853-a4e3-4d9d-834a-8fff5116bd0b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:49:55.372556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c3b853-a4e3-4d9d-834a-8fff5116bd0b service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:49:55.378930 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a6fa4-cded-4b81-94fd-837559ab25ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:49:55.379552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6fa4-cded-4b81-94fd-837559ab25ca service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:49:55.379836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6fa4-cded-4b81-94fd-837559ab25ca service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:49:55.380124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a6fa4-cded-4b81-94fd-837559ab25ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:49:55.382100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-378a6fa4-cded-4b81-94fd-837559ab25ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:49:55.393671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a6fa4-cded-4b81-94fd-837559ab25ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:49:55.395287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a6fa4-cded-4b81-94fd-837559ab25ca service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:49:55.405214 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1210/2386] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:49:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:50:05.401277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:51:44.059633 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e960886-7d58-4853-8063-aa5371c42775 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:51:44.070578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e960886-7d58-4853-8063-aa5371c42775 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:51:44.075581 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d152d-86c6-4aaf-985d-26546ca03a19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:51:44.076196 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d152d-86c6-4aaf-985d-26546ca03a19 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:51:44.076476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d152d-86c6-4aaf-985d-26546ca03a19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:51:44.076744 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d152d-86c6-4aaf-985d-26546ca03a19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:51:44.086952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d152d-86c6-4aaf-985d-26546ca03a19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:51:44.088379 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d152d-86c6-4aaf-985d-26546ca03a19 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:51:44.098277 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1177/2387] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:51:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:51:44.187373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a6fa4-cded-4b81-94fd-837559ab25ca service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:51:44.197295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a6fa4-cded-4b81-94fd-837559ab25ca service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:51:44.201688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8464ed-215f-4013-b6df-00b071419ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:51:44.202349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8464ed-215f-4013-b6df-00b071419ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:51:44.202668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8464ed-215f-4013-b6df-00b071419ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:51:44.202913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8464ed-215f-4013-b6df-00b071419ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:51:44.204844 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7d8464ed-215f-4013-b6df-00b071419ac5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:51:44.280733 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8464ed-215f-4013-b6df-00b071419ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:51:44.282211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8464ed-215f-4013-b6df-00b071419ac5 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:51:44.291471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1211/2388] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:51:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:51:44.344464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d152d-86c6-4aaf-985d-26546ca03a19 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:51:44.356675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d152d-86c6-4aaf-985d-26546ca03a19 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:51:44.361991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b34ceb-1e5a-4e81-b607-34a2d060e119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:51:44.362609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b34ceb-1e5a-4e81-b607-34a2d060e119 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:51:44.362886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b34ceb-1e5a-4e81-b607-34a2d060e119 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:51:44.363172 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b34ceb-1e5a-4e81-b607-34a2d060e119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:51:44.365481 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-06b34ceb-1e5a-4e81-b607-34a2d060e119 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:51:44.462869 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b34ceb-1e5a-4e81-b607-34a2d060e119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:51:44.464807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b34ceb-1e5a-4e81-b607-34a2d060e119 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:51:44.484568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1178/2389] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:51:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:51:54.287469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=33 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:51:54.481780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=47 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:52:55.099837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8464ed-215f-4013-b6df-00b071419ac5 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:52:55.111828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8464ed-215f-4013-b6df-00b071419ac5 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:52:55.118021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d95d8e-5b75-4da3-bc62-50a725fd8979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:52:55.118652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d95d8e-5b75-4da3-bc62-50a725fd8979 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:52:55.118929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d95d8e-5b75-4da3-bc62-50a725fd8979 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:52:55.119196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d95d8e-5b75-4da3-bc62-50a725fd8979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:52:55.127705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d95d8e-5b75-4da3-bc62-50a725fd8979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:52:55.129273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d95d8e-5b75-4da3-bc62-50a725fd8979 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:52:55.138045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1212/2390] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:52:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:53:05.134327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:53:44.644824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b34ceb-1e5a-4e81-b607-34a2d060e119 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:53:44.654002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b34ceb-1e5a-4e81-b607-34a2d060e119 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:53:44.658799 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d396a9-49ae-40d4-b110-cd5628feb251 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:53:44.659595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d396a9-49ae-40d4-b110-cd5628feb251 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:53:44.659874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d396a9-49ae-40d4-b110-cd5628feb251 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:53:44.660167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d396a9-49ae-40d4-b110-cd5628feb251 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:53:44.669774 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d396a9-49ae-40d4-b110-cd5628feb251 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:53:44.671246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d396a9-49ae-40d4-b110-cd5628feb251 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:53:44.680901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1179/2391] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:53:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:53:54.676433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:55:01.333596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d95d8e-5b75-4da3-bc62-50a725fd8979 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:01.366405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d95d8e-5b75-4da3-bc62-50a725fd8979 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:01.371931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ce07d-c4b0-49f5-8725-e96e8e8e038f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:01.372553 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ce07d-c4b0-49f5-8725-e96e8e8e038f service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:01.372861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ce07d-c4b0-49f5-8725-e96e8e8e038f service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:01.373132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ce07d-c4b0-49f5-8725-e96e8e8e038f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:01.382392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ce07d-c4b0-49f5-8725-e96e8e8e038f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:01.383991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ce07d-c4b0-49f5-8725-e96e8e8e038f service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:01.394180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1213/2392] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:55:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:55:11.390126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=13 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:55:45.515048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d396a9-49ae-40d4-b110-cd5628feb251 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:45.535169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d396a9-49ae-40d4-b110-cd5628feb251 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:45.539454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37247c-fccd-4bad-9890-ab6ab41ad942 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:45.540080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37247c-fccd-4bad-9890-ab6ab41ad942 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:45.540363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37247c-fccd-4bad-9890-ab6ab41ad942 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:45.540637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e37247c-fccd-4bad-9890-ab6ab41ad942 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:45.542767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e37247c-fccd-4bad-9890-ab6ab41ad942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:45.670823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37247c-fccd-4bad-9890-ab6ab41ad942 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:45.672180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e37247c-fccd-4bad-9890-ab6ab41ad942 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:45.692617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1180/2393] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:55:48.816299 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ce07d-c4b0-49f5-8725-e96e8e8e038f service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:48.869187 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ce07d-c4b0-49f5-8725-e96e8e8e038f service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:48.877417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286d8e9-dbd8-4ce5-b7de-f58fbb8a723f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:48.878084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286d8e9-dbd8-4ce5-b7de-f58fbb8a723f service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:48.878585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286d8e9-dbd8-4ce5-b7de-f58fbb8a723f service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:48.878864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6286d8e9-dbd8-4ce5-b7de-f58fbb8a723f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:48.881236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6286d8e9-dbd8-4ce5-b7de-f58fbb8a723f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:48.888233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37247c-fccd-4bad-9890-ab6ab41ad942 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:48.889772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5e37247c-fccd-4bad-9890-ab6ab41ad942 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:48.896809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286d8e9-dbd8-4ce5-b7de-f58fbb8a723f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:48.898621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6286d8e9-dbd8-4ce5-b7de-f58fbb8a723f service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:48.902767 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e37247c-fccd-4bad-9890-ab6ab41ad942 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:48.907454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:48.908110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:48.908395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:48.908679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] PATH_INFO: `/v3/users/08c4ec6a688c4608aa75aaf046933eec` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:48.913456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1214/2394] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:55:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:55:48.923102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08c4ec6a688c4608aa75aaf046933eec)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:48.925053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:48.972038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08c4ec6a688c4608aa75aaf046933eec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:55:48.972712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08c4ec6a688c4608aa75aaf046933eec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:55:48.984729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08c4ec6a688c4608aa75aaf046933eec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:55:48.989321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:55:48.992135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08c4ec6a688c4608aa75aaf046933eec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:55:49.002932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cf4e111d-6842-4321-abdd-09ad66abe173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08c4ec6a688c4608aa75aaf046933eec'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:55:49.009150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1181/2395] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:55:48 2022] DELETE /identity/v3/users/08c4ec6a688c4608aa75aaf046933eec => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 07:55:49.188371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286d8e9-dbd8-4ce5-b7de-f58fbb8a723f service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:49.200240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6286d8e9-dbd8-4ce5-b7de-f58fbb8a723f service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:49.260542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:49.261197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:49.261478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:49.261763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] PATH_INFO: `/v3/projects/0874292e3f8f4db98adc2b9672f840c4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:49.272230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0874292e3f8f4db98adc2b9672f840c4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:49.274228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:49.325838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0874292e3f8f4db98adc2b9672f840c4', 'request_id': 'req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:55:49.332389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0874292e3f8f4db98adc2b9672f840c4', 'request_id': 'req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:55:49.338090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0874292e3f8f4db98adc2b9672f840c4', 'request_id': 'req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:55:49.352353 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] The token cache is being invalidate because project 0874292e3f8f4db98adc2b9672f840c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:55:49.352654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:55:49.353268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-4898b1ca-f4cf-4c41-97f1-c3caf5bb8c14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:55:49.354862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1215/2396] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:55:49 2022] DELETE /identity/v3/projects/0874292e3f8f4db98adc2b9672f840c4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:55:49.362464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6471-2907-4d51-b726-7577c82ec6d4 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:49.362749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6471-2907-4d51-b726-7577c82ec6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:49.363109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d6471-2907-4d51-b726-7577c82ec6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:49.430555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fc3d6471-2907-4d51-b726-7577c82ec6d4 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:55:49.431827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc3d6471-2907-4d51-b726-7577c82ec6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:49.494328 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1182/2397] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:55:49.502247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66111bad-0795-4262-af15-55446a9c303f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:49.502523 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66111bad-0795-4262-af15-55446a9c303f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:49.502804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66111bad-0795-4262-af15-55446a9c303f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:49.523340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-66111bad-0795-4262-af15-55446a9c303f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:55:49.524541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-66111bad-0795-4262-af15-55446a9c303f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:49.532819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1216/2398] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:55:49.537158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d6471-2907-4d51-b726-7577c82ec6d4 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:49.547095 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d6471-2907-4d51-b726-7577c82ec6d4 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:49.551655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668de448-eed3-424f-a304-b467f4658509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:49.552271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668de448-eed3-424f-a304-b467f4658509 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:49.552551 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668de448-eed3-424f-a304-b467f4658509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:49.552824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668de448-eed3-424f-a304-b467f4658509 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:49.553525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668de448-eed3-424f-a304-b467f4658509 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:55:49.553869 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668de448-eed3-424f-a304-b467f4658509 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:49.555470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668de448-eed3-424f-a304-b467f4658509 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:49.567323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1183/2399] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 07:55:49.571683 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66111bad-0795-4262-af15-55446a9c303f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:49.581895 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66111bad-0795-4262-af15-55446a9c303f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:49.587906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cafbfa-f964-4f38-9072-d53040e72a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:49.588521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cafbfa-f964-4f38-9072-d53040e72a08 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:49.588801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cafbfa-f964-4f38-9072-d53040e72a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:49.589069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cafbfa-f964-4f38-9072-d53040e72a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:49.589778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cafbfa-f964-4f38-9072-d53040e72a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:49.591729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cafbfa-f964-4f38-9072-d53040e72a08 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:49.624749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:55:49.632059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1217/2400] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:55:49 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:55:49.637536 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668de448-eed3-424f-a304-b467f4658509 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:49.647880 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668de448-eed3-424f-a304-b467f4658509 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:49.704672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ab055-70df-49a4-ba42-65ba03cd533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:49.705291 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ab055-70df-49a4-ba42-65ba03cd533f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:49.705571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ab055-70df-49a4-ba42-65ba03cd533f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:49.705879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ab055-70df-49a4-ba42-65ba03cd533f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:49.706547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ab055-70df-49a4-ba42-65ba03cd533f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:49.708464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ab055-70df-49a4-ba42-65ba03cd533f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:49.731137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1184/2401] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:55:49 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 07:55:49.735408 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cafbfa-f964-4f38-9072-d53040e72a08 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:49.746319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cafbfa-f964-4f38-9072-d53040e72a08 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:49.752157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c27a5b-44c0-43ed-be9d-cedf8e999fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:49.752818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c27a5b-44c0-43ed-be9d-cedf8e999fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:49.753098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c27a5b-44c0-43ed-be9d-cedf8e999fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:49.753370 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c27a5b-44c0-43ed-be9d-cedf8e999fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:49.754151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c27a5b-44c0-43ed-be9d-cedf8e999fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:49.755784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c27a5b-44c0-43ed-be9d-cedf8e999fb6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:49.765971 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1218/2402] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:55:49.770431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ab055-70df-49a4-ba42-65ba03cd533f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:49.785376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ab055-70df-49a4-ba42-65ba03cd533f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:49.793040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400e832-3c36-48d3-bf39-f00fa41cc24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:49.793941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400e832-3c36-48d3-bf39-f00fa41cc24c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:49.794237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400e832-3c36-48d3-bf39-f00fa41cc24c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:49.794585 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400e832-3c36-48d3-bf39-f00fa41cc24c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:49.795738 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400e832-3c36-48d3-bf39-f00fa41cc24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:49.797342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400e832-3c36-48d3-bf39-f00fa41cc24c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:49.808375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1185/2403] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:55:49.812837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c27a5b-44c0-43ed-be9d-cedf8e999fb6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:49.823389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c27a5b-44c0-43ed-be9d-cedf8e999fb6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:49.829230 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146cf24-7c01-4a5c-ab5c-2b3fcef49ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:49.829955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146cf24-7c01-4a5c-ab5c-2b3fcef49ff8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:49.830242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146cf24-7c01-4a5c-ab5c-2b3fcef49ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:49.830539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146cf24-7c01-4a5c-ab5c-2b3fcef49ff8 admin admin] PATH_INFO: `/v3/projects/0898dc56ea6a416aae031efb7fefc46b/users/dc97d25991d24cf9bb84f49366c895a6/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:49.848660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146cf24-7c01-4a5c-ab5c-2b3fcef49ff8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0898dc56ea6a416aae031efb7fefc46b, user_id=dc97d25991d24cf9bb84f49366c895a6, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:49.852684 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146cf24-7c01-4a5c-ab5c-2b3fcef49ff8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:49.870842 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1219/2404] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:55:49 2022] PUT /identity/v3/projects/0898dc56ea6a416aae031efb7fefc46b/users/dc97d25991d24cf9bb84f49366c895a6/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:55:49.878465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501232-b13a-451f-a70b-40a6c7e7c5af None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:49.878761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501232-b13a-451f-a70b-40a6c7e7c5af None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:49.879039 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09501232-b13a-451f-a70b-40a6c7e7c5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:49.902346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-09501232-b13a-451f-a70b-40a6c7e7c5af None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:55:49.903782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-09501232-b13a-451f-a70b-40a6c7e7c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:49.927018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:55:49.975445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1186/2405] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:55:49.985890 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146cf24-7c01-4a5c-ab5c-2b3fcef49ff8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:49.987807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4146cf24-7c01-4a5c-ab5c-2b3fcef49ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:50.048227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146cf24-7c01-4a5c-ab5c-2b3fcef49ff8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:50.052009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f472f47-b52d-4818-9dfe-0c6be4709ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:50.052643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f472f47-b52d-4818-9dfe-0c6be4709ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:50.052918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f472f47-b52d-4818-9dfe-0c6be4709ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:50.053191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f472f47-b52d-4818-9dfe-0c6be4709ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:50.063351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f472f47-b52d-4818-9dfe-0c6be4709ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:50.064792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f472f47-b52d-4818-9dfe-0c6be4709ca6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:50.075268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1220/2406] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:55:53.720241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e818479-05a8-4f03-9eb4-37535f233452 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:53.720781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e818479-05a8-4f03-9eb4-37535f233452 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:53.720781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e818479-05a8-4f03-9eb4-37535f233452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:53.748982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0e818479-05a8-4f03-9eb4-37535f233452 None None] MFA Rules not processed for user `dc97d25991d24cf9bb84f49366c895a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:55:53.750273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0e818479-05a8-4f03-9eb4-37535f233452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:53.819808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1187/2407] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:55:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:55:53.835606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f472f47-b52d-4818-9dfe-0c6be4709ca6 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:53.837172 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f472f47-b52d-4818-9dfe-0c6be4709ca6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:53.936507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f472f47-b52d-4818-9dfe-0c6be4709ca6 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:53.942638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46094b9-85a7-4110-bb12-9a2addc2babf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:53.943301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46094b9-85a7-4110-bb12-9a2addc2babf service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:53.943581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46094b9-85a7-4110-bb12-9a2addc2babf service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:53.943855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46094b9-85a7-4110-bb12-9a2addc2babf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:53.952771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46094b9-85a7-4110-bb12-9a2addc2babf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:53.954422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46094b9-85a7-4110-bb12-9a2addc2babf service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:53.963708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1221/2408] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:55:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:55:55.322280 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03efc84-46be-4c72-92e4-b8d729d6b5f5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:55.323183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03efc84-46be-4c72-92e4-b8d729d6b5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:55.323183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03efc84-46be-4c72-92e4-b8d729d6b5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:55.343192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c03efc84-46be-4c72-92e4-b8d729d6b5f5 None None] MFA Rules not processed for user `dc97d25991d24cf9bb84f49366c895a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:55:55.344388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c03efc84-46be-4c72-92e4-b8d729d6b5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:55.352534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1188/2409] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:55:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:55:55.393755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46094b9-85a7-4110-bb12-9a2addc2babf service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:55.395290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f46094b9-85a7-4110-bb12-9a2addc2babf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:55.458816 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:55:55.489826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46094b9-85a7-4110-bb12-9a2addc2babf service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:55.495259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5747013-55e3-40bd-95f7-f3f3293fab0d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c6943686c843e588df14f7c6dffb8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:55.495922 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5747013-55e3-40bd-95f7-f3f3293fab0d service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:55.496267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5747013-55e3-40bd-95f7-f3f3293fab0d service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:55.496756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5747013-55e3-40bd-95f7-f3f3293fab0d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:55.498985 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b5747013-55e3-40bd-95f7-f3f3293fab0d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:55.516829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5747013-55e3-40bd-95f7-f3f3293fab0d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:55.518456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5747013-55e3-40bd-95f7-f3f3293fab0d service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:55.530686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1222/2410] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:55:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:55:55.811377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03efc84-46be-4c72-92e4-b8d729d6b5f5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:55.821389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03efc84-46be-4c72-92e4-b8d729d6b5f5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:55.825893 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016f19-e5bc-48c6-a1a1-1c1da6fbe865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:55.826514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016f19-e5bc-48c6-a1a1-1c1da6fbe865 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:55.826810 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016f19-e5bc-48c6-a1a1-1c1da6fbe865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:55.827195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2016f19-e5bc-48c6-a1a1-1c1da6fbe865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:55.829012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e2016f19-e5bc-48c6-a1a1-1c1da6fbe865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:55.838822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2016f19-e5bc-48c6-a1a1-1c1da6fbe865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:55.840232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2016f19-e5bc-48c6-a1a1-1c1da6fbe865 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:55.848217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1189/2411] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:55:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:55:56.432497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5747013-55e3-40bd-95f7-f3f3293fab0d service glance] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:56.434195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b5747013-55e3-40bd-95f7-f3f3293fab0d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:56.445093 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016f19-e5bc-48c6-a1a1-1c1da6fbe865 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:56.447110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e2016f19-e5bc-48c6-a1a1-1c1da6fbe865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:56.458704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2016f19-e5bc-48c6-a1a1-1c1da6fbe865 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:56.463191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ce805-ece6-4ba7-8c38-2fc288662c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:56.463875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ce805-ece6-4ba7-8c38-2fc288662c44 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:56.464350 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ce805-ece6-4ba7-8c38-2fc288662c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:56.464826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ce805-ece6-4ba7-8c38-2fc288662c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:56.467046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f91ce805-ece6-4ba7-8c38-2fc288662c44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:56.482480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ce805-ece6-4ba7-8c38-2fc288662c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:56.484291 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ce805-ece6-4ba7-8c38-2fc288662c44 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:56.495978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1190/2412] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:55:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:55:56.519336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5747013-55e3-40bd-95f7-f3f3293fab0d service glance] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:56.524944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff51a0a-4a96-4666-b34c-1cc6f2d88e17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:56.525678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff51a0a-4a96-4666-b34c-1cc6f2d88e17 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:56.526120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff51a0a-4a96-4666-b34c-1cc6f2d88e17 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:56.526378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff51a0a-4a96-4666-b34c-1cc6f2d88e17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:56.529127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fff51a0a-4a96-4666-b34c-1cc6f2d88e17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:56.546430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff51a0a-4a96-4666-b34c-1cc6f2d88e17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:56.548057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff51a0a-4a96-4666-b34c-1cc6f2d88e17 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:56.564850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1223/2413] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:55:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:55:59.079404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ce805-ece6-4ba7-8c38-2fc288662c44 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:55:59.080973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f91ce805-ece6-4ba7-8c38-2fc288662c44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:59.101145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ce805-ece6-4ba7-8c38-2fc288662c44 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:55:59.106868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717689a-9cad-4cec-b361-16540ee2b48a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:55:59.107574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717689a-9cad-4cec-b361-16540ee2b48a service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:55:59.107861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717689a-9cad-4cec-b361-16540ee2b48a service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:55:59.108189 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717689a-9cad-4cec-b361-16540ee2b48a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:55:59.110093 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6717689a-9cad-4cec-b361-16540ee2b48a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:55:59.124596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717689a-9cad-4cec-b361-16540ee2b48a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:55:59.126344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6717689a-9cad-4cec-b361-16540ee2b48a service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:55:59.140017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1191/2414] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:55:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:56:02.459235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff51a0a-4a96-4666-b34c-1cc6f2d88e17 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:56:02.473347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff51a0a-4a96-4666-b34c-1cc6f2d88e17 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:56:02.478781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf62a07-a9b7-4ffa-8c83-e26f7d89d887 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:56:02.479490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf62a07-a9b7-4ffa-8c83-e26f7d89d887 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:56:02.479768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf62a07-a9b7-4ffa-8c83-e26f7d89d887 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:56:02.480032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf62a07-a9b7-4ffa-8c83-e26f7d89d887 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:56:02.481789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ccf62a07-a9b7-4ffa-8c83-e26f7d89d887 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:56:02.494260 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf62a07-a9b7-4ffa-8c83-e26f7d89d887 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:56:02.495839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf62a07-a9b7-4ffa-8c83-e26f7d89d887 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:56:02.506520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1224/2415] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:56:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:56:09.135395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=70 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:56:12.502154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=56 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:57:45.369115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717689a-9cad-4cec-b361-16540ee2b48a service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:57:45.383029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717689a-9cad-4cec-b361-16540ee2b48a service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:57:45.388803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84247-d8a4-429e-9a8b-8353ec9d6ce9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:57:45.389444 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84247-d8a4-429e-9a8b-8353ec9d6ce9 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:57:45.389856 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84247-d8a4-429e-9a8b-8353ec9d6ce9 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:57:45.390040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b84247-d8a4-429e-9a8b-8353ec9d6ce9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:57:45.392198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-36b84247-d8a4-429e-9a8b-8353ec9d6ce9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:57:45.419775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b84247-d8a4-429e-9a8b-8353ec9d6ce9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:57:45.421378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b84247-d8a4-429e-9a8b-8353ec9d6ce9 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:57:45.434962 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1192/2416] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:57:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 07:57:45.568578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf62a07-a9b7-4ffa-8c83-e26f7d89d887 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:57:45.570221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ccf62a07-a9b7-4ffa-8c83-e26f7d89d887 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:57:45.677580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf62a07-a9b7-4ffa-8c83-e26f7d89d887 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:57:45.681261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634f0db-1ed0-467d-92ca-7c981be5e154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:57:45.681990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634f0db-1ed0-467d-92ca-7c981be5e154 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:57:45.682271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634f0db-1ed0-467d-92ca-7c981be5e154 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:57:45.682592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634f0db-1ed0-467d-92ca-7c981be5e154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:57:45.684271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6634f0db-1ed0-467d-92ca-7c981be5e154 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:57:45.746573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634f0db-1ed0-467d-92ca-7c981be5e154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:57:45.748036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634f0db-1ed0-467d-92ca-7c981be5e154 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:57:45.757864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1225/2417] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:57:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:57:45.824057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84247-d8a4-429e-9a8b-8353ec9d6ce9 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:57:45.835749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b84247-d8a4-429e-9a8b-8353ec9d6ce9 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:57:45.840718 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55457e12-b8e4-409c-bd5e-f684663ddabe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:57:45.841642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457e12-b8e4-409c-bd5e-f684663ddabe service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:57:45.842002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457e12-b8e4-409c-bd5e-f684663ddabe service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:57:45.842662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55457e12-b8e4-409c-bd5e-f684663ddabe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:57:45.845611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55457e12-b8e4-409c-bd5e-f684663ddabe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:57:45.928748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55457e12-b8e4-409c-bd5e-f684663ddabe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:57:45.930233 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55457e12-b8e4-409c-bd5e-f684663ddabe service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:57:45.950058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1193/2418] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:57:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:57:48.674282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c535107-8e25-4d6e-a081-0ba29c8c16ba None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:57:48.674820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c535107-8e25-4d6e-a081-0ba29c8c16ba None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:57:48.674820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c535107-8e25-4d6e-a081-0ba29c8c16ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:57:48.695456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0c535107-8e25-4d6e-a081-0ba29c8c16ba None None] MFA Rules not processed for user `dc97d25991d24cf9bb84f49366c895a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:57:48.696678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0c535107-8e25-4d6e-a081-0ba29c8c16ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:57:48.704023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1226/2419] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:57:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:57:49.018095 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55457e12-b8e4-409c-bd5e-f684663ddabe service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:57:49.038029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55457e12-b8e4-409c-bd5e-f684663ddabe service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:57:49.041492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb7d7e-ee9f-449f-ad0b-98e6ff7281cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:57:49.042208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb7d7e-ee9f-449f-ad0b-98e6ff7281cb service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:57:49.042489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb7d7e-ee9f-449f-ad0b-98e6ff7281cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:57:49.042766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eb7d7e-ee9f-449f-ad0b-98e6ff7281cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:57:49.044533 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-54eb7d7e-ee9f-449f-ad0b-98e6ff7281cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:57:49.068103 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb7d7e-ee9f-449f-ad0b-98e6ff7281cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:57:49.069421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54eb7d7e-ee9f-449f-ad0b-98e6ff7281cb service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:57:49.078297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1194/2420] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:57:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:57:57.208759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c535107-8e25-4d6e-a081-0ba29c8c16ba None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:57:57.219145 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c535107-8e25-4d6e-a081-0ba29c8c16ba None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:57:57.223134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf47724-0e55-4e92-832c-846af4d13dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:57:57.223747 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf47724-0e55-4e92-832c-846af4d13dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:57:57.224021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf47724-0e55-4e92-832c-846af4d13dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:57:57.224302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf47724-0e55-4e92-832c-846af4d13dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:57:57.226177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-acf47724-0e55-4e92-832c-846af4d13dd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:57:57.236117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf47724-0e55-4e92-832c-846af4d13dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:57:57.237546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf47724-0e55-4e92-832c-846af4d13dd8 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:57:57.246115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1227/2421] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:57:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:57:59.074656 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=62 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:58:07.242229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=66 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:59:38.282502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb7d7e-ee9f-449f-ad0b-98e6ff7281cb service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:38.296556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eb7d7e-ee9f-449f-ad0b-98e6ff7281cb service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:38.301924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23cad6-f629-4291-a910-a63b6a474db8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:38.302565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23cad6-f629-4291-a910-a63b6a474db8 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:38.302847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23cad6-f629-4291-a910-a63b6a474db8 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:38.303124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d23cad6-f629-4291-a910-a63b6a474db8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:38.304917 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d23cad6-f629-4291-a910-a63b6a474db8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:59:38.349509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23cad6-f629-4291-a910-a63b6a474db8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:38.349509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d23cad6-f629-4291-a910-a63b6a474db8 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:38.350936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf47724-0e55-4e92-832c-846af4d13dd8 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:38.362437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1195/2422] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:59:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:59:38.363169 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf47724-0e55-4e92-832c-846af4d13dd8 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:38.367730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:38.368368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:38.368881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:38.369166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] PATH_INFO: `/v3/users/dc97d25991d24cf9bb84f49366c895a6` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:38.370336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc97d25991d24cf9bb84f49366c895a6)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:38.372275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:38.419572 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc97d25991d24cf9bb84f49366c895a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:59:38.420224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc97d25991d24cf9bb84f49366c895a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:59:38.432148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc97d25991d24cf9bb84f49366c895a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:59:38.439085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc97d25991d24cf9bb84f49366c895a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:59:38.447352 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-8a1960b8-05e7-4a41-904f-939913cf5286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc97d25991d24cf9bb84f49366c895a6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:59:38.453534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1228/2423] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 07:59:38 2022] DELETE /identity/v3/users/dc97d25991d24cf9bb84f49366c895a6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:59:38.627199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23cad6-f629-4291-a910-a63b6a474db8 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:38.638972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d23cad6-f629-4291-a910-a63b6a474db8 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:38.696478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:38.697122 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:38.697398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:38.697673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] PATH_INFO: `/v3/projects/0898dc56ea6a416aae031efb7fefc46b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:38.698916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0898dc56ea6a416aae031efb7fefc46b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:38.700821 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:38.747988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0898dc56ea6a416aae031efb7fefc46b', 'request_id': 'req-80d0964a-2f77-4e42-930c-454c7c0d2c9b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:59:38.753386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0898dc56ea6a416aae031efb7fefc46b', 'request_id': 'req-80d0964a-2f77-4e42-930c-454c7c0d2c9b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:59:38.761763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0898dc56ea6a416aae031efb7fefc46b', 'request_id': 'req-80d0964a-2f77-4e42-930c-454c7c0d2c9b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:59:38.772850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] The token cache is being invalidate because project 0898dc56ea6a416aae031efb7fefc46b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 07:59:38.773145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:59:38.773778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-80d0964a-2f77-4e42-930c-454c7c0d2c9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 07:59:38.775532 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1196/2424] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 07:59:38 2022] DELETE /identity/v3/projects/0898dc56ea6a416aae031efb7fefc46b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:59:38.781216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d59c-8b7f-42f7-8dbf-f50b956e6441 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:38.781531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d59c-8b7f-42f7-8dbf-f50b956e6441 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:38.781838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2d59c-8b7f-42f7-8dbf-f50b956e6441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:38.801604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c3c2d59c-8b7f-42f7-8dbf-f50b956e6441 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:59:38.802934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c3c2d59c-8b7f-42f7-8dbf-f50b956e6441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:59:38.865894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1229/2425] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:59:38.873775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802487c-b7c2-4a81-9518-d8295d721af6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:38.874104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802487c-b7c2-4a81-9518-d8295d721af6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:38.874380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802487c-b7c2-4a81-9518-d8295d721af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:38.896260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b802487c-b7c2-4a81-9518-d8295d721af6 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:59:38.897462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b802487c-b7c2-4a81-9518-d8295d721af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:59:38.905881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1197/2426] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:59:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:59:38.910177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2d59c-8b7f-42f7-8dbf-f50b956e6441 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:38.919802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2d59c-8b7f-42f7-8dbf-f50b956e6441 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:38.925094 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391d440-7f97-49e3-a6a6-d7d820c52747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:38.925705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391d440-7f97-49e3-a6a6-d7d820c52747 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:38.926020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391d440-7f97-49e3-a6a6-d7d820c52747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:38.926293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a391d440-7f97-49e3-a6a6-d7d820c52747 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:38.926989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391d440-7f97-49e3-a6a6-d7d820c52747 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 07:59:38.927294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391d440-7f97-49e3-a6a6-d7d820c52747 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:38.928901 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a391d440-7f97-49e3-a6a6-d7d820c52747 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:38.940020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1230/2427] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 07:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 07:59:38.944109 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b802487c-b7c2-4a81-9518-d8295d721af6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:38.953545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b802487c-b7c2-4a81-9518-d8295d721af6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:38.958401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d71e0-f503-4b95-9fcf-735ee1b5b25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:38.958998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d71e0-f503-4b95-9fcf-735ee1b5b25a admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:38.959275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d71e0-f503-4b95-9fcf-735ee1b5b25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:38.959547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d71e0-f503-4b95-9fcf-735ee1b5b25a admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:38.960217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d71e0-f503-4b95-9fcf-735ee1b5b25a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:38.962139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d71e0-f503-4b95-9fcf-735ee1b5b25a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:38.990252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1198/2428] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 07:59:38 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:59:38.994388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391d440-7f97-49e3-a6a6-d7d820c52747 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:39.004414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a391d440-7f97-49e3-a6a6-d7d820c52747 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:39.064366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e4c95-710c-47cd-af45-afe7f9a587aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:39.064997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e4c95-710c-47cd-af45-afe7f9a587aa admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:39.065505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e4c95-710c-47cd-af45-afe7f9a587aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:39.065826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0e4c95-710c-47cd-af45-afe7f9a587aa admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:39.066486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e4c95-710c-47cd-af45-afe7f9a587aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:39.068412 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0e4c95-710c-47cd-af45-afe7f9a587aa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:39.077001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:59:39.095724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1231/2429] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 07:59:38 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 07:59:39.097068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d71e0-f503-4b95-9fcf-735ee1b5b25a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:39.108029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d71e0-f503-4b95-9fcf-735ee1b5b25a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:39.113973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e94189-836a-40ea-ac05-bfcce28b387a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:39.114588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e94189-836a-40ea-ac05-bfcce28b387a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:39.114854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e94189-836a-40ea-ac05-bfcce28b387a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:39.115124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e94189-836a-40ea-ac05-bfcce28b387a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:39.115823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e94189-836a-40ea-ac05-bfcce28b387a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:39.117473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e94189-836a-40ea-ac05-bfcce28b387a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:39.128588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1199/2430] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:59:39.132911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e4c95-710c-47cd-af45-afe7f9a587aa admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:39.143417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0e4c95-710c-47cd-af45-afe7f9a587aa admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:39.148148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd9ffe-fce1-4227-9fe6-022e8f8e9d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:39.148761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd9ffe-fce1-4227-9fe6-022e8f8e9d90 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:39.149038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd9ffe-fce1-4227-9fe6-022e8f8e9d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:39.149306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd9ffe-fce1-4227-9fe6-022e8f8e9d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:39.150084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd9ffe-fce1-4227-9fe6-022e8f8e9d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:39.151693 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbd9ffe-fce1-4227-9fe6-022e8f8e9d90 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:39.162540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1232/2431] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 07:59:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 07:59:39.166877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e94189-836a-40ea-ac05-bfcce28b387a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:39.177153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e94189-836a-40ea-ac05-bfcce28b387a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:39.184024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402b0f4-6876-497f-b382-63f4fc7cc57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:39.184700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402b0f4-6876-497f-b382-63f4fc7cc57a admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:39.184984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402b0f4-6876-497f-b382-63f4fc7cc57a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:39.185282 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402b0f4-6876-497f-b382-63f4fc7cc57a admin admin] PATH_INFO: `/v3/projects/ae75dfeb5dfd43899ae3d9c3d7504915/users/3e0463a987a34884a845a0f7e2d82fa9/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:39.202603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402b0f4-6876-497f-b382-63f4fc7cc57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae75dfeb5dfd43899ae3d9c3d7504915, user_id=3e0463a987a34884a845a0f7e2d82fa9, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:39.205312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402b0f4-6876-497f-b382-63f4fc7cc57a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:39.219586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1200/2432] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 07:59:39 2022] PUT /identity/v3/projects/ae75dfeb5dfd43899ae3d9c3d7504915/users/3e0463a987a34884a845a0f7e2d82fa9/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 07:59:39.229899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc74256-9194-4737-8d62-002da488e540 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:39.230212 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc74256-9194-4737-8d62-002da488e540 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:39.230484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc74256-9194-4737-8d62-002da488e540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:39.259019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8dc74256-9194-4737-8d62-002da488e540 None None] MFA Rules not processed for user `3e0463a987a34884a845a0f7e2d82fa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 07:59:39.260204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8dc74256-9194-4737-8d62-002da488e540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:59:39.327764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1233/2433] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 07:59:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 07:59:46.409423 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402b0f4-6876-497f-b382-63f4fc7cc57a admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:46.412658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a402b0f4-6876-497f-b382-63f4fc7cc57a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:59:46.422530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:59:46.478569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402b0f4-6876-497f-b382-63f4fc7cc57a admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:46.482977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e78fc-438f-4fb6-b58f-82ad8c955419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:46.483600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e78fc-438f-4fb6-b58f-82ad8c955419 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:46.483880 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e78fc-438f-4fb6-b58f-82ad8c955419 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:46.484160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e78fc-438f-4fb6-b58f-82ad8c955419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:46.486213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-235e78fc-438f-4fb6-b58f-82ad8c955419 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 07:59:46.567220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e78fc-438f-4fb6-b58f-82ad8c955419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:46.568658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e78fc-438f-4fb6-b58f-82ad8c955419 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:46.591785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1201/2434] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:59:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:59:46.979527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc74256-9194-4737-8d62-002da488e540 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 07:59:46.989633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc74256-9194-4737-8d62-002da488e540 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 07:59:46.994041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f00d90-bf0d-42b6-b8c2-91a82daca54f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 07:59:46.994621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f00d90-bf0d-42b6-b8c2-91a82daca54f service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 07:59:46.995071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f00d90-bf0d-42b6-b8c2-91a82daca54f service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 07:59:46.995444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f00d90-bf0d-42b6-b8c2-91a82daca54f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 07:59:47.004822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f00d90-bf0d-42b6-b8c2-91a82daca54f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 07:59:47.006352 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f00d90-bf0d-42b6-b8c2-91a82daca54f service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 07:59:47.015270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1234/2435] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 07:59:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 07:59:56.589083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=60 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 07:59:57.011439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=3,keystone:SELECT=45 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:11.327860 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e78fc-438f-4fb6-b58f-82ad8c955419 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:11.329639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-235e78fc-438f-4fb6-b58f-82ad8c955419 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:11.402040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e78fc-438f-4fb6-b58f-82ad8c955419 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:11.408366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88458d-5a87-4cb5-87b5-1d8a7c9e02a5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:11.408996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88458d-5a87-4cb5-87b5-1d8a7c9e02a5 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:11.409279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88458d-5a87-4cb5-87b5-1d8a7c9e02a5 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:11.409554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88458d-5a87-4cb5-87b5-1d8a7c9e02a5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:11.411606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ab88458d-5a87-4cb5-87b5-1d8a7c9e02a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:11.424467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88458d-5a87-4cb5-87b5-1d8a7c9e02a5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:11.426275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88458d-5a87-4cb5-87b5-1d8a7c9e02a5 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:11.441302 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1202/2436] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:20.419414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f00d90-bf0d-42b6-b8c2-91a82daca54f service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:20.421797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-71f00d90-bf0d-42b6-b8c2-91a82daca54f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:20.492085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f00d90-bf0d-42b6-b8c2-91a82daca54f service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:20.495715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a41ee-e22f-457a-8dbe-8aeeca8f66ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c6943686c843e588df14f7c6dffb8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:20.496546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a41ee-e22f-457a-8dbe-8aeeca8f66ee service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:20.496854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a41ee-e22f-457a-8dbe-8aeeca8f66ee service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:20.497295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a41ee-e22f-457a-8dbe-8aeeca8f66ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:20.499284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-038a41ee-e22f-457a-8dbe-8aeeca8f66ee service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application [None req-038a41ee-e22f-457a-8dbe-8aeeca8f66ee service glance] Could not find project: c80c8785a6f7482caf8f72d11937d27f.: keystone.exception.ProjectNotFound: Could not find project: c80c8785a6f7482caf8f72d11937d27f. Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jul 26 08:00:20.512136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 08:00:20.513436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self._validate_token_resources() Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 26 08:00:20.514877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.get_or_create( Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application with Lock( Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._enter() Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.creator() Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application created_value = creator( Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 26 08:00:20.516162 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 26 08:00:20.517363 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: c80c8785a6f7482caf8f72d11937d27f. Jul 26 08:00:20.517363 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: ERROR keystone.server.flask.application Jul 26 08:00:20.517363 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1235/2437] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:20 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 94 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Jul 26 08:00:21.434967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=32 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:21.593218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88458d-5a87-4cb5-87b5-1d8a7c9e02a5 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:21.596003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ab88458d-5a87-4cb5-87b5-1d8a7c9e02a5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:21.618178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a41ee-e22f-457a-8dbe-8aeeca8f66ee service glance] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:21.620627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-038a41ee-e22f-457a-8dbe-8aeeca8f66ee service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:21.712256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88458d-5a87-4cb5-87b5-1d8a7c9e02a5 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:21.718012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345566d-b8d4-463c-9ff5-ca10ad654bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:21.719704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345566d-b8d4-463c-9ff5-ca10ad654bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:21.720274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345566d-b8d4-463c-9ff5-ca10ad654bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:21.720781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345566d-b8d4-463c-9ff5-ca10ad654bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:21.726409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c345566d-b8d4-463c-9ff5-ca10ad654bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:21.756764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a41ee-e22f-457a-8dbe-8aeeca8f66ee service glance] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:21.764979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9f2b4-8a71-474d-9a5f-59f6c405f2d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:21.765626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9f2b4-8a71-474d-9a5f-59f6c405f2d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:21.765931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9f2b4-8a71-474d-9a5f-59f6c405f2d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:21.766475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc9f2b4-8a71-474d-9a5f-59f6c405f2d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:21.796041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-edc9f2b4-8a71-474d-9a5f-59f6c405f2d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:21.831004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345566d-b8d4-463c-9ff5-ca10ad654bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:21.832615 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345566d-b8d4-463c-9ff5-ca10ad654bb6 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:21.856704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1203/2438] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:21.913851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9f2b4-8a71-474d-9a5f-59f6c405f2d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:21.916014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc9f2b4-8a71-474d-9a5f-59f6c405f2d6 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:21.941341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1236/2439] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:00:23.095331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345566d-b8d4-463c-9ff5-ca10ad654bb6 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:23.097034 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c345566d-b8d4-463c-9ff5-ca10ad654bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:23.115157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345566d-b8d4-463c-9ff5-ca10ad654bb6 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:23.124794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:23.125715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:23.126388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:23.126877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] PATH_INFO: `/v3/users/c394476504af4c1d8bea462a5113cebc` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:23.129153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c394476504af4c1d8bea462a5113cebc)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:23.131901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:23.202492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c394476504af4c1d8bea462a5113cebc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:23.203428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c394476504af4c1d8bea462a5113cebc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:23.220590 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c394476504af4c1d8bea462a5113cebc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:23.243724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c394476504af4c1d8bea462a5113cebc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:23.261838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f77f7331-96cf-4696-a567-b259978eb595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c394476504af4c1d8bea462a5113cebc'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:23.278272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1204/2440] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:23 2022] DELETE /identity/v3/users/c394476504af4c1d8bea462a5113cebc => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:23.594451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9f2b4-8a71-474d-9a5f-59f6c405f2d6 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:23.633665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc9f2b4-8a71-474d-9a5f-59f6c405f2d6 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:23.692853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:23.700223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:23.700873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:23.701152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:23.701420 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] PATH_INFO: `/v3/projects/6fc12ef80ee841f5af4bc2d56966cadc` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:23.702664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fc12ef80ee841f5af4bc2d56966cadc)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:23.704658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:23.754836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fc12ef80ee841f5af4bc2d56966cadc', 'request_id': 'req-3c5e055f-f786-4298-bbac-34e63cdd5cd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:23.761853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fc12ef80ee841f5af4bc2d56966cadc', 'request_id': 'req-3c5e055f-f786-4298-bbac-34e63cdd5cd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:23.768685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fc12ef80ee841f5af4bc2d56966cadc', 'request_id': 'req-3c5e055f-f786-4298-bbac-34e63cdd5cd4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:23.781513 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] The token cache is being invalidate because project 6fc12ef80ee841f5af4bc2d56966cadc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:23.781831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:23.782447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3c5e055f-f786-4298-bbac-34e63cdd5cd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:23.783985 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1237/2441] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:23 2022] DELETE /identity/v3/projects/6fc12ef80ee841f5af4bc2d56966cadc => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:23.789802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcefe8-ea4d-4c46-98dc-b89d81cb7bf1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:23.790096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcefe8-ea4d-4c46-98dc-b89d81cb7bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:23.790374 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fcefe8-ea4d-4c46-98dc-b89d81cb7bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:23.810353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-36fcefe8-ea4d-4c46-98dc-b89d81cb7bf1 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:23.811668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-36fcefe8-ea4d-4c46-98dc-b89d81cb7bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:23.829271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:23.887250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1205/2442] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:23.896034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6bece-d515-41a7-8cec-373206c94c35 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:23.896157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6bece-d515-41a7-8cec-373206c94c35 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:23.896434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6bece-d515-41a7-8cec-373206c94c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:23.916755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-abc6bece-d515-41a7-8cec-373206c94c35 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:23.918221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-abc6bece-d515-41a7-8cec-373206c94c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:23.926136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1238/2443] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:23.931076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcefe8-ea4d-4c46-98dc-b89d81cb7bf1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:23.943540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fcefe8-ea4d-4c46-98dc-b89d81cb7bf1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:23.950973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e1aa4-a4b5-4795-a76e-e446df909965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:23.951812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e1aa4-a4b5-4795-a76e-e446df909965 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:23.952492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e1aa4-a4b5-4795-a76e-e446df909965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:23.952870 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e1aa4-a4b5-4795-a76e-e446df909965 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:23.953915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e1aa4-a4b5-4795-a76e-e446df909965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:23.954466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e1aa4-a4b5-4795-a76e-e446df909965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:23.957049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e1aa4-a4b5-4795-a76e-e446df909965 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:23.973151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1206/2444] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:00:23.976926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6bece-d515-41a7-8cec-373206c94c35 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:23.988048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6bece-d515-41a7-8cec-373206c94c35 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:23.993050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586282f5-fed6-4d44-b28e-502b72619e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:23.993678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586282f5-fed6-4d44-b28e-502b72619e11 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:23.993989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586282f5-fed6-4d44-b28e-502b72619e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:23.994263 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586282f5-fed6-4d44-b28e-502b72619e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:23.994931 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586282f5-fed6-4d44-b28e-502b72619e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:23.996872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586282f5-fed6-4d44-b28e-502b72619e11 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:24.030080 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1239/2445] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:23 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:00:24.034224 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e1aa4-a4b5-4795-a76e-e446df909965 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:24.045017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e1aa4-a4b5-4795-a76e-e446df909965 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:24.121838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4835cd8-c760-4b4d-bd26-404654765e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:24.124953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4835cd8-c760-4b4d-bd26-404654765e32 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:24.125420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4835cd8-c760-4b4d-bd26-404654765e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:24.125909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4835cd8-c760-4b4d-bd26-404654765e32 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:24.127044 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4835cd8-c760-4b4d-bd26-404654765e32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:24.130021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4835cd8-c760-4b4d-bd26-404654765e32 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:24.159995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1207/2446] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:24 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:24.163804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586282f5-fed6-4d44-b28e-502b72619e11 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:24.177730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586282f5-fed6-4d44-b28e-502b72619e11 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:24.185921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadeee91-d692-4d05-94ac-8cace090d77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:24.187608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadeee91-d692-4d05-94ac-8cace090d77f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:24.188088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadeee91-d692-4d05-94ac-8cace090d77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:24.188685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadeee91-d692-4d05-94ac-8cace090d77f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:24.190091 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadeee91-d692-4d05-94ac-8cace090d77f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:24.192627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadeee91-d692-4d05-94ac-8cace090d77f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:24.206810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1240/2447] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:24.210621 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4835cd8-c760-4b4d-bd26-404654765e32 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:24.226878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4835cd8-c760-4b4d-bd26-404654765e32 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:24.232496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e65a92f-375c-4eea-84d3-196cd9a2f0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:24.233977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65a92f-375c-4eea-84d3-196cd9a2f0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:24.234264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65a92f-375c-4eea-84d3-196cd9a2f0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:24.234730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e65a92f-375c-4eea-84d3-196cd9a2f0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:24.235643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65a92f-375c-4eea-84d3-196cd9a2f0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:24.237606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e65a92f-375c-4eea-84d3-196cd9a2f0f8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:24.250094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1208/2448] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:24.254145 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadeee91-d692-4d05-94ac-8cace090d77f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:24.266088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadeee91-d692-4d05-94ac-8cace090d77f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:24.274438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bd930-ad77-425f-aeea-72ad62688c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:24.275311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bd930-ad77-425f-aeea-72ad62688c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:24.275688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bd930-ad77-425f-aeea-72ad62688c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:24.276086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bd930-ad77-425f-aeea-72ad62688c6a admin admin] PATH_INFO: `/v3/projects/8a46af50be774973800f747cdaf66896/users/dc6c885fee5440799beac8cfd3e42822/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:24.295390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bd930-ad77-425f-aeea-72ad62688c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a46af50be774973800f747cdaf66896, user_id=dc6c885fee5440799beac8cfd3e42822, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:24.296803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bd930-ad77-425f-aeea-72ad62688c6a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:24.316157 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1241/2449] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:24 2022] PUT /identity/v3/projects/8a46af50be774973800f747cdaf66896/users/dc6c885fee5440799beac8cfd3e42822/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:24.321483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca171e-626d-4afa-b5bb-1c11b629508a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:24.321804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca171e-626d-4afa-b5bb-1c11b629508a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:24.322087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca171e-626d-4afa-b5bb-1c11b629508a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:24.341080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-69ca171e-626d-4afa-b5bb-1c11b629508a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:24.342391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69ca171e-626d-4afa-b5bb-1c11b629508a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:24.412721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1209/2450] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:24.424091 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bd930-ad77-425f-aeea-72ad62688c6a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:24.426012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fc9bd930-ad77-425f-aeea-72ad62688c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:24.504270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bd930-ad77-425f-aeea-72ad62688c6a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:24.508084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8116b-5686-4f28-ac60-b62f94d97e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:24.508794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8116b-5686-4f28-ac60-b62f94d97e97 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:24.509158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8116b-5686-4f28-ac60-b62f94d97e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:24.509501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af8116b-5686-4f28-ac60-b62f94d97e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:24.522447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8116b-5686-4f28-ac60-b62f94d97e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:24.524065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af8116b-5686-4f28-ac60-b62f94d97e97 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:24.536440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1242/2451] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:26.660664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca171e-626d-4afa-b5bb-1c11b629508a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:26.662190 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69ca171e-626d-4afa-b5bb-1c11b629508a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:26.677492 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca171e-626d-4afa-b5bb-1c11b629508a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:26.688480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:26.689167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:26.689453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:26.689752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] PATH_INFO: `/v3/users/57029844165f4d83892176fe82594685` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:26.691001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57029844165f4d83892176fe82594685)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:26.692980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:26.743161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:26.768977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8116b-5686-4f28-ac60-b62f94d97e97 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:26.770766 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8af8116b-5686-4f28-ac60-b62f94d97e97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:26.786175 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af8116b-5686-4f28-ac60-b62f94d97e97 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:26.790264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b41fcb-d841-47df-9779-bf588f190873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:26.791026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b41fcb-d841-47df-9779-bf588f190873 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:26.791405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b41fcb-d841-47df-9779-bf588f190873 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:26.791772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b41fcb-d841-47df-9779-bf588f190873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:26.795289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-60b41fcb-d841-47df-9779-bf588f190873 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:26.795924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57029844165f4d83892176fe82594685'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:26.798440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57029844165f4d83892176fe82594685'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:26.835076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57029844165f4d83892176fe82594685'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:26.846912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57029844165f4d83892176fe82594685'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:26.848107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b41fcb-d841-47df-9779-bf588f190873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:26.850415 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b41fcb-d841-47df-9779-bf588f190873 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:26.859363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-188a04e9-2572-4672-b84d-1d5a2e8d2d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57029844165f4d83892176fe82594685'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:26.865696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1243/2452] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:26.870020 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1210/2453] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:26 2022] DELETE /identity/v3/users/57029844165f4d83892176fe82594685 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:27.659975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b41fcb-d841-47df-9779-bf588f190873 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:27.681318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b41fcb-d841-47df-9779-bf588f190873 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:27.733623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:27.857065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:27.858799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:27.859253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:27.859840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] PATH_INFO: `/v3/projects/4fa0c5574ef9497ebe29658f8164764c` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:27.861461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa0c5574ef9497ebe29658f8164764c)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:27.864087 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:27.968584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa0c5574ef9497ebe29658f8164764c', 'request_id': 'req-e853c59c-8262-4588-9610-be746fa2891b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:27.984032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa0c5574ef9497ebe29658f8164764c', 'request_id': 'req-e853c59c-8262-4588-9610-be746fa2891b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:28.004136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa0c5574ef9497ebe29658f8164764c', 'request_id': 'req-e853c59c-8262-4588-9610-be746fa2891b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:28.042354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] The token cache is being invalidate because project 4fa0c5574ef9497ebe29658f8164764c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:28.044307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:28.047617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e853c59c-8262-4588-9610-be746fa2891b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:28.050769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1244/2454] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:27 2022] DELETE /identity/v3/projects/4fa0c5574ef9497ebe29658f8164764c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:28.057571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0891fcd-4fc6-4645-a33d-9b6813f873e9 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:28.058153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0891fcd-4fc6-4645-a33d-9b6813f873e9 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:28.058281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0891fcd-4fc6-4645-a33d-9b6813f873e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:28.108672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c0891fcd-4fc6-4645-a33d-9b6813f873e9 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:28.116260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c0891fcd-4fc6-4645-a33d-9b6813f873e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:28.221728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1211/2455] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:28.230799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985c152-3955-4d78-936d-89d2837ab73c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:28.231177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985c152-3955-4d78-936d-89d2837ab73c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:28.231903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985c152-3955-4d78-936d-89d2837ab73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:28.266211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1985c152-3955-4d78-936d-89d2837ab73c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:28.269605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1985c152-3955-4d78-936d-89d2837ab73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:28.292294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0891fcd-4fc6-4645-a33d-9b6813f873e9 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:28.294781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1245/2456] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:28.308120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0891fcd-4fc6-4645-a33d-9b6813f873e9 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:28.325861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd169-95a2-4f47-b763-c4ea74c0740f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:28.326686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd169-95a2-4f47-b763-c4ea74c0740f admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:28.327061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd169-95a2-4f47-b763-c4ea74c0740f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:28.327606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffd169-95a2-4f47-b763-c4ea74c0740f admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:28.328609 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffd169-95a2-4f47-b763-c4ea74c0740f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:28.329242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffd169-95a2-4f47-b763-c4ea74c0740f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:28.330991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffd169-95a2-4f47-b763-c4ea74c0740f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:28.378681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1212/2457] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 08:00:28.379910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985c152-3955-4d78-936d-89d2837ab73c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:28.401921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985c152-3955-4d78-936d-89d2837ab73c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:28.409531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ba21b-39a4-41de-9195-4ad30d59c4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:28.410276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ba21b-39a4-41de-9195-4ad30d59c4c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:28.410501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ba21b-39a4-41de-9195-4ad30d59c4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:28.410836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3ba21b-39a4-41de-9195-4ad30d59c4c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:28.411986 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ba21b-39a4-41de-9195-4ad30d59c4c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:28.414191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3ba21b-39a4-41de-9195-4ad30d59c4c0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:28.480725 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1246/2458] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:28 2022] POST /identity/v3/projects => generated 363 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:28.482040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd169-95a2-4f47-b763-c4ea74c0740f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:28.495490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffd169-95a2-4f47-b763-c4ea74c0740f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:28.576957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7fbcf-da7b-41f9-baca-99bc572a3926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:28.577856 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7fbcf-da7b-41f9-baca-99bc572a3926 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:28.578146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7fbcf-da7b-41f9-baca-99bc572a3926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:28.578569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7fbcf-da7b-41f9-baca-99bc572a3926 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:28.579399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7fbcf-da7b-41f9-baca-99bc572a3926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:28.582311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7fbcf-da7b-41f9-baca-99bc572a3926 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:28.624784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1213/2459] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:28 2022] POST /identity/v3/users => generated 282 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:28.628857 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ba21b-39a4-41de-9195-4ad30d59c4c0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:28.642977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3ba21b-39a4-41de-9195-4ad30d59c4c0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:28.648185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb3179-84c8-408a-ac2b-48de019f3dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:28.648940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb3179-84c8-408a-ac2b-48de019f3dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:28.649333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb3179-84c8-408a-ac2b-48de019f3dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:28.649699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb3179-84c8-408a-ac2b-48de019f3dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:28.650677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb3179-84c8-408a-ac2b-48de019f3dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:28.652626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb3179-84c8-408a-ac2b-48de019f3dbf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:28.672029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1247/2460] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:28.678083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7fbcf-da7b-41f9-baca-99bc572a3926 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:28.692451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7fbcf-da7b-41f9-baca-99bc572a3926 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:28.698473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dbebb3-5823-490c-9d06-aee5e3adc95e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:28.699420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbebb3-5823-490c-9d06-aee5e3adc95e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:28.699903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbebb3-5823-490c-9d06-aee5e3adc95e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:28.700426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dbebb3-5823-490c-9d06-aee5e3adc95e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:28.701763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbebb3-5823-490c-9d06-aee5e3adc95e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:28.706556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dbebb3-5823-490c-9d06-aee5e3adc95e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:28.724203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1214/2461] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:28.726773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb3179-84c8-408a-ac2b-48de019f3dbf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:28.738882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb3179-84c8-408a-ac2b-48de019f3dbf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:28.745554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04906847-48d1-47ea-98a1-bf3e2a2b0a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:28.746283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04906847-48d1-47ea-98a1-bf3e2a2b0a58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:28.746566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04906847-48d1-47ea-98a1-bf3e2a2b0a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:28.746837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04906847-48d1-47ea-98a1-bf3e2a2b0a58 admin admin] PATH_INFO: `/v3/projects/fec9baec71844377959528f6027f88c1/users/cb68426698274d99a7f156317a869d06/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:28.764702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04906847-48d1-47ea-98a1-bf3e2a2b0a58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fec9baec71844377959528f6027f88c1, user_id=cb68426698274d99a7f156317a869d06, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:28.767464 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04906847-48d1-47ea-98a1-bf3e2a2b0a58 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:28.786292 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1248/2462] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:28 2022] PUT /identity/v3/projects/fec9baec71844377959528f6027f88c1/users/cb68426698274d99a7f156317a869d06/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:28.798826 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484782fd-04f6-493a-912a-e0b778657c82 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:28.799027 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484782fd-04f6-493a-912a-e0b778657c82 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:28.799271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484782fd-04f6-493a-912a-e0b778657c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:28.814965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:28.835874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-484782fd-04f6-493a-912a-e0b778657c82 None None] MFA Rules not processed for user `cb68426698274d99a7f156317a869d06`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:28.837727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-484782fd-04f6-493a-912a-e0b778657c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:28.919331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1215/2463] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:28 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:28.924097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04906847-48d1-47ea-98a1-bf3e2a2b0a58 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:28.977992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:29.002906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04906847-48d1-47ea-98a1-bf3e2a2b0a58 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:29.007966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9533340-f40a-4c41-9c11-7e49e38b8636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:29.008991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9533340-f40a-4c41-9c11-7e49e38b8636 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:29.009482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9533340-f40a-4c41-9c11-7e49e38b8636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:29.009839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9533340-f40a-4c41-9c11-7e49e38b8636 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:29.010468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9533340-f40a-4c41-9c11-7e49e38b8636 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:29.012587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9533340-f40a-4c41-9c11-7e49e38b8636 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:29.047514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1249/2464] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:28 2022] POST /identity/v3/projects => generated 363 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:00:29.050951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484782fd-04f6-493a-912a-e0b778657c82 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:29.066195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484782fd-04f6-493a-912a-e0b778657c82 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:29.148544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adab2bc-5e62-46bf-b258-312c98e907aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:29.149309 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adab2bc-5e62-46bf-b258-312c98e907aa admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:29.149602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adab2bc-5e62-46bf-b258-312c98e907aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:29.149962 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adab2bc-5e62-46bf-b258-312c98e907aa admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:29.150706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adab2bc-5e62-46bf-b258-312c98e907aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:29.152747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adab2bc-5e62-46bf-b258-312c98e907aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:29.181861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1216/2465] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:29 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:00:29.182880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9533340-f40a-4c41-9c11-7e49e38b8636 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:29.202110 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9533340-f40a-4c41-9c11-7e49e38b8636 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:29.207587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaad2db-84b2-4084-a0b3-4b8eff7d7468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:29.208264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaad2db-84b2-4084-a0b3-4b8eff7d7468 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:29.208607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaad2db-84b2-4084-a0b3-4b8eff7d7468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:29.208891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaad2db-84b2-4084-a0b3-4b8eff7d7468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:29.209874 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaad2db-84b2-4084-a0b3-4b8eff7d7468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:29.211519 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaad2db-84b2-4084-a0b3-4b8eff7d7468 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:29.224598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1250/2466] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:29.229846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adab2bc-5e62-46bf-b258-312c98e907aa admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:29.242279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adab2bc-5e62-46bf-b258-312c98e907aa admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:29.249067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f97ce-67e2-4be6-9b04-d145b105dbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:29.249797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f97ce-67e2-4be6-9b04-d145b105dbc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:29.250084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f97ce-67e2-4be6-9b04-d145b105dbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:29.250360 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f97ce-67e2-4be6-9b04-d145b105dbc5 admin admin] PATH_INFO: `/v3/projects/8880e3b458c0405fb28ca3f9fd64ac37/users/bf809bae7d7349348a1b9bd26743bdd3/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:29.271854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f97ce-67e2-4be6-9b04-d145b105dbc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8880e3b458c0405fb28ca3f9fd64ac37, user_id=bf809bae7d7349348a1b9bd26743bdd3, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:29.274714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f97ce-67e2-4be6-9b04-d145b105dbc5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:29.292073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1217/2467] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:29 2022] PUT /identity/v3/projects/8880e3b458c0405fb28ca3f9fd64ac37/users/bf809bae7d7349348a1b9bd26743bdd3/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:29.296629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaad2db-84b2-4084-a0b3-4b8eff7d7468 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:29.309487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaad2db-84b2-4084-a0b3-4b8eff7d7468 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:29.417853 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f97ce-67e2-4be6-9b04-d145b105dbc5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:29.419830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1f2f97ce-67e2-4be6-9b04-d145b105dbc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:29.449616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b99aed-f97c-411f-b5cf-937697551b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:29.450468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b99aed-f97c-411f-b5cf-937697551b23 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:29.450752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b99aed-f97c-411f-b5cf-937697551b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:29.451023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b99aed-f97c-411f-b5cf-937697551b23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:29.455077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b99aed-f97c-411f-b5cf-937697551b23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:29.456750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b99aed-f97c-411f-b5cf-937697551b23 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:29.500279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1251/2468] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:29 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:00:29.504809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b99aed-f97c-411f-b5cf-937697551b23 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:29.523008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b99aed-f97c-411f-b5cf-937697551b23 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:29.538167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc5d413-90e3-4fa7-8229-7c5b2c1eb6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:29.540457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d413-90e3-4fa7-8229-7c5b2c1eb6f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:29.543171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d413-90e3-4fa7-8229-7c5b2c1eb6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:29.543726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc5d413-90e3-4fa7-8229-7c5b2c1eb6f0 admin admin] PATH_INFO: `/v3/projects/8880e3b458c0405fb28ca3f9fd64ac37/users/bf809bae7d7349348a1b9bd26743bdd3/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:29.546718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5d413-90e3-4fa7-8229-7c5b2c1eb6f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8880e3b458c0405fb28ca3f9fd64ac37, user_id=bf809bae7d7349348a1b9bd26743bdd3, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:29.557374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc5d413-90e3-4fa7-8229-7c5b2c1eb6f0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:29.567039 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f97ce-67e2-4be6-9b04-d145b105dbc5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:29.594517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1252/2469] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:29 2022] PUT /identity/v3/projects/8880e3b458c0405fb28ca3f9fd64ac37/users/bf809bae7d7349348a1b9bd26743bdd3/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:29.600543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde47287-6642-4373-bf18-1c344baa5d51 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:29.600977 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde47287-6642-4373-bf18-1c344baa5d51 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:29.601377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde47287-6642-4373-bf18-1c344baa5d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:29.611831 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:29.684490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-fde47287-6642-4373-bf18-1c344baa5d51 None None] MFA Rules not processed for user `bf809bae7d7349348a1b9bd26743bdd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:29.686212 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fde47287-6642-4373-bf18-1c344baa5d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:29.727836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:29.729698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:29.731219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:29.733500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] PATH_INFO: `/v3/users/3e0463a987a34884a845a0f7e2d82fa9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:29.746711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e0463a987a34884a845a0f7e2d82fa9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:29.748797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:29.854650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1253/2470] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:29 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:29.858767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde47287-6642-4373-bf18-1c344baa5d51 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:29.860861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fde47287-6642-4373-bf18-1c344baa5d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:29.861601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e0463a987a34884a845a0f7e2d82fa9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:29.863259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e0463a987a34884a845a0f7e2d82fa9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:29.895655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e0463a987a34884a845a0f7e2d82fa9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:29.906967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e0463a987a34884a845a0f7e2d82fa9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:29.913321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:29.927969 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-23138826-2621-4da8-9f32-b7dcf9c4cdcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e0463a987a34884a845a0f7e2d82fa9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:29.938365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1218/2471] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:29 2022] DELETE /identity/v3/users/3e0463a987a34884a845a0f7e2d82fa9 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:00:29.942594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ae0a8-849c-4075-b163-e7c48dad4451 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:29.943002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ae0a8-849c-4075-b163-e7c48dad4451 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:29.943279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ae0a8-849c-4075-b163-e7c48dad4451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:29.985410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c52ae0a8-849c-4075-b163-e7c48dad4451 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:29.986710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c52ae0a8-849c-4075-b163-e7c48dad4451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:29.990306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde47287-6642-4373-bf18-1c344baa5d51 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:30.094800 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9c2ee-13f4-4533-bc10-f1558365538e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:30.095668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9c2ee-13f4-4533-bc10-f1558365538e service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:30.095999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9c2ee-13f4-4533-bc10-f1558365538e service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:30.097420 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9c2ee-13f4-4533-bc10-f1558365538e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:30.100921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5ee9c2ee-13f4-4533-bc10-f1558365538e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:30.120239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1219/2472] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:30.132941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ae0a8-849c-4075-b163-e7c48dad4451 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:30.135315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c52ae0a8-849c-4075-b163-e7c48dad4451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:30.197680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:30.250517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ae0a8-849c-4075-b163-e7c48dad4451 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:30.252521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9c2ee-13f4-4533-bc10-f1558365538e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:30.254407 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee9c2ee-13f4-4533-bc10-f1558365538e service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:30.256263 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdb260-70a7-4cca-981c-4fa8ecad8d32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:30.257632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdb260-70a7-4cca-981c-4fa8ecad8d32 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:30.258007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdb260-70a7-4cca-981c-4fa8ecad8d32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:30.258404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cdb260-70a7-4cca-981c-4fa8ecad8d32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:30.272082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdb260-70a7-4cca-981c-4fa8ecad8d32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:30.273613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cdb260-70a7-4cca-981c-4fa8ecad8d32 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:30.274782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1254/2473] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:29 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:30.287941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1220/2474] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:00:31.043746 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9c2ee-13f4-4533-bc10-f1558365538e service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:31.045758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5ee9c2ee-13f4-4533-bc10-f1558365538e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:31.075348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee9c2ee-13f4-4533-bc10-f1558365538e service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:31.085265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:31.086353 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:31.086872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:31.087539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] PATH_INFO: `/v3/users/cb68426698274d99a7f156317a869d06` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:31.090517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb68426698274d99a7f156317a869d06)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:31.093338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:31.246082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdb260-70a7-4cca-981c-4fa8ecad8d32 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:31.248003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb68426698274d99a7f156317a869d06'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:31.248645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b6cdb260-70a7-4cca-981c-4fa8ecad8d32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:31.254267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb68426698274d99a7f156317a869d06'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:31.277384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cdb260-70a7-4cca-981c-4fa8ecad8d32 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:31.288128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:31.289754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:31.290798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:31.291660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] PATH_INFO: `/v3/projects/ae75dfeb5dfd43899ae3d9c3d7504915` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:31.293261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb68426698274d99a7f156317a869d06'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:31.295237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae75dfeb5dfd43899ae3d9c3d7504915)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:31.300137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:31.311034 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb68426698274d99a7f156317a869d06'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:31.330835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-35e4e941-65a2-4452-b53e-561e4c97b0ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb68426698274d99a7f156317a869d06'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:31.334144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:31.342584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1255/2475] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:31 2022] DELETE /identity/v3/users/cb68426698274d99a7f156317a869d06 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:31.347675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25026a52-8517-40ae-a3da-4348681cabd6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:31.348867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25026a52-8517-40ae-a3da-4348681cabd6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:31.349979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25026a52-8517-40ae-a3da-4348681cabd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:31.402610 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae75dfeb5dfd43899ae3d9c3d7504915', 'request_id': 'req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:31.411932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-25026a52-8517-40ae-a3da-4348681cabd6 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:31.421573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae75dfeb5dfd43899ae3d9c3d7504915', 'request_id': 'req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:31.421934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-25026a52-8517-40ae-a3da-4348681cabd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:31.441460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae75dfeb5dfd43899ae3d9c3d7504915', 'request_id': 'req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:31.465137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] The token cache is being invalidate because project ae75dfeb5dfd43899ae3d9c3d7504915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:31.465532 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:31.466764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6ba7e592-abe0-40b2-ac4a-da2e43e3ee06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:31.475013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1221/2476] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:31 2022] DELETE /identity/v3/projects/ae75dfeb5dfd43899ae3d9c3d7504915 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:31.480821 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d0e5-7248-4fd6-a905-24376d757e82 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:31.482246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d0e5-7248-4fd6-a905-24376d757e82 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:31.483176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3d0e5-7248-4fd6-a905-24376d757e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:31.537823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-79a3d0e5-7248-4fd6-a905-24376d757e82 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:31.539367 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-79a3d0e5-7248-4fd6-a905-24376d757e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:31.609354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1256/2477] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:31.626305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25026a52-8517-40ae-a3da-4348681cabd6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:31.627789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-25026a52-8517-40ae-a3da-4348681cabd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:31.696251 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1222/2478] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:31.706682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a247f-2c1c-40cd-ac9f-a245a49b2714 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:31.706924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a247f-2c1c-40cd-ac9f-a245a49b2714 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:31.707205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a247f-2c1c-40cd-ac9f-a245a49b2714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:31.737701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-4b7a247f-2c1c-40cd-ac9f-a245a49b2714 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:31.745091 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4b7a247f-2c1c-40cd-ac9f-a245a49b2714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:31.758303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1223/2479] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:31.759698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a247f-2c1c-40cd-ac9f-a245a49b2714 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:31.776315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a247f-2c1c-40cd-ac9f-a245a49b2714 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:31.784992 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25026a52-8517-40ae-a3da-4348681cabd6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:31.790496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e91237-f8f1-4171-8c3b-48e7ebe14b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:31.791160 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e91237-f8f1-4171-8c3b-48e7ebe14b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:31.791441 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e91237-f8f1-4171-8c3b-48e7ebe14b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:31.791712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e91237-f8f1-4171-8c3b-48e7ebe14b31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:31.792433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e91237-f8f1-4171-8c3b-48e7ebe14b31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:31.792746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e91237-f8f1-4171-8c3b-48e7ebe14b31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:31.794509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e91237-f8f1-4171-8c3b-48e7ebe14b31 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:31.816369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2949a3-6687-48bc-b918-fdcf45257930 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:31.817355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2949a3-6687-48bc-b918-fdcf45257930 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:31.817355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2949a3-6687-48bc-b918-fdcf45257930 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:31.817573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2949a3-6687-48bc-b918-fdcf45257930 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:31.821646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1224/2480] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:00:31.838742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e91237-f8f1-4171-8c3b-48e7ebe14b31 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:31.847444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2949a3-6687-48bc-b918-fdcf45257930 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:31.849006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2949a3-6687-48bc-b918-fdcf45257930 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:31.863968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e91237-f8f1-4171-8c3b-48e7ebe14b31 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:31.874529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1257/2481] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:31.875269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4e946-9ea9-4ac5-b0d8-91664f046ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:31.876006 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4e946-9ea9-4ac5-b0d8-91664f046ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:31.876268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4e946-9ea9-4ac5-b0d8-91664f046ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:31.876714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4e946-9ea9-4ac5-b0d8-91664f046ae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:31.877211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4e946-9ea9-4ac5-b0d8-91664f046ae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:31.880759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4e946-9ea9-4ac5-b0d8-91664f046ae6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:31.956098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1225/2482] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:31 2022] POST /identity/v3/projects => generated 405 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:31.961438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2949a3-6687-48bc-b918-fdcf45257930 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:31.983839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2949a3-6687-48bc-b918-fdcf45257930 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.029654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752bf3e-a203-4745-9423-5febd00a18ea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.029983 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752bf3e-a203-4745-9423-5febd00a18ea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.030236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752bf3e-a203-4745-9423-5febd00a18ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.081336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:32.087488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a752bf3e-a203-4745-9423-5febd00a18ea None None] MFA Rules not processed for user `dc6c885fee5440799beac8cfd3e42822`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:32.088732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a752bf3e-a203-4745-9423-5febd00a18ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.091377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e7cb8-06bb-4cc3-97ba-1a6910989aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.092046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e7cb8-06bb-4cc3-97ba-1a6910989aae admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.092329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e7cb8-06bb-4cc3-97ba-1a6910989aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.096760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e7cb8-06bb-4cc3-97ba-1a6910989aae admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.097498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e7cb8-06bb-4cc3-97ba-1a6910989aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.100071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288e7cb8-06bb-4cc3-97ba-1a6910989aae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.148259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1258/2483] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:31 2022] POST /identity/v3/users => generated 303 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:32.149509 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e7cb8-06bb-4cc3-97ba-1a6910989aae admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.152420 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-288e7cb8-06bb-4cc3-97ba-1a6910989aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.166436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288e7cb8-06bb-4cc3-97ba-1a6910989aae admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.190133 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.191277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.191714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.192144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] PATH_INFO: `/v3/users/886151888fc84ccf890d609ba62da9e1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.194342 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=886151888fc84ccf890d609ba62da9e1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.197941 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.222167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1226/2484] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:32.223072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752bf3e-a203-4745-9423-5febd00a18ea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.241995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:32.243194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a752bf3e-a203-4745-9423-5febd00a18ea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.256194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ad804-5d66-4166-809f-c1496c1f85a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.256901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ad804-5d66-4166-809f-c1496c1f85a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.257038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ad804-5d66-4166-809f-c1496c1f85a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.257326 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ad804-5d66-4166-809f-c1496c1f85a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.258038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ad804-5d66-4166-809f-c1496c1f85a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.259831 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6ad804-5d66-4166-809f-c1496c1f85a1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.270947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1227/2485] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:32.271985 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ad804-5d66-4166-809f-c1496c1f85a1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.274319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-dc6ad804-5d66-4166-809f-c1496c1f85a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.275555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '886151888fc84ccf890d609ba62da9e1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.277908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '886151888fc84ccf890d609ba62da9e1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.288320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ad804-5d66-4166-809f-c1496c1f85a1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.294335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611daa1-444e-4460-9bc8-8f0dfafc6d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.294976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611daa1-444e-4460-9bc8-8f0dfafc6d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.295256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611daa1-444e-4460-9bc8-8f0dfafc6d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.295561 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8611daa1-444e-4460-9bc8-8f0dfafc6d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.296242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611daa1-444e-4460-9bc8-8f0dfafc6d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.298193 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8611daa1-444e-4460-9bc8-8f0dfafc6d47 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.304018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '886151888fc84ccf890d609ba62da9e1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.311861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '886151888fc84ccf890d609ba62da9e1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.325096 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '886151888fc84ccf890d609ba62da9e1'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.336066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1259/2486] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:32 2022] DELETE /identity/v3/users/886151888fc84ccf890d609ba62da9e1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:32.337332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.345448 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1228/2487] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:32 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:32.345663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.347924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611daa1-444e-4460-9bc8-8f0dfafc6d47 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.353852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8611daa1-444e-4460-9bc8-8f0dfafc6d47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.453693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8611daa1-444e-4460-9bc8-8f0dfafc6d47 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.454794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86bf80c-9d34-40fe-853a-52d186ded6f5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.458934 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc341e0-032d-46ed-adc9-9ebca54715b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.459738 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc341e0-032d-46ed-adc9-9ebca54715b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.459881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f272c-bbd8-465a-a593-f9e7a565c61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.460084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc341e0-032d-46ed-adc9-9ebca54715b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.460851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc341e0-032d-46ed-adc9-9ebca54715b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.460975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f272c-bbd8-465a-a593-f9e7a565c61a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.460975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f272c-bbd8-465a-a593-f9e7a565c61a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.461149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f272c-bbd8-465a-a593-f9e7a565c61a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.461231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc341e0-032d-46ed-adc9-9ebca54715b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.461713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f272c-bbd8-465a-a593-f9e7a565c61a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.463261 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc341e0-032d-46ed-adc9-9ebca54715b3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.463405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f272c-bbd8-465a-a593-f9e7a565c61a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.479495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1260/2488] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:32.480573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f272c-bbd8-465a-a593-f9e7a565c61a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.483466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-bd8f272c-bbd8-465a-a593-f9e7a565c61a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.495777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1229/2489] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:32 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:00:32.497227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc341e0-032d-46ed-adc9-9ebca54715b3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.504161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f272c-bbd8-465a-a593-f9e7a565c61a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.509201 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.509926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.510266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.510545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] PATH_INFO: `/v3/projects/fec9baec71844377959528f6027f88c1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.511220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc341e0-032d-46ed-adc9-9ebca54715b3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.511900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fec9baec71844377959528f6027f88c1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.514174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.519133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8d3e2-22c0-4f1d-bc7d-ddf9e03d9d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.519942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8d3e2-22c0-4f1d-bc7d-ddf9e03d9d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.520229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8d3e2-22c0-4f1d-bc7d-ddf9e03d9d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.520505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8d3e2-22c0-4f1d-bc7d-ddf9e03d9d72 admin admin] PATH_INFO: `/v3/projects/2ca9bdb96092498fb692fb58deab254a/users/b1fa01f22f73426ea7910abb63ebd1aa/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.552087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8d3e2-22c0-4f1d-bc7d-ddf9e03d9d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ca9bdb96092498fb692fb58deab254a, user_id=b1fa01f22f73426ea7910abb63ebd1aa, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.556341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8d3e2-22c0-4f1d-bc7d-ddf9e03d9d72 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.575517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fec9baec71844377959528f6027f88c1', 'request_id': 'req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.578526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1230/2490] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:32 2022] PUT /identity/v3/projects/2ca9bdb96092498fb692fb58deab254a/users/b1fa01f22f73426ea7910abb63ebd1aa/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:32.584577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fec9baec71844377959528f6027f88c1', 'request_id': 'req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.588670 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8d3e2-22c0-4f1d-bc7d-ddf9e03d9d72 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.598398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fec9baec71844377959528f6027f88c1', 'request_id': 'req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.614471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] The token cache is being invalidate because project fec9baec71844377959528f6027f88c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:32.614769 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.616089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.618862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1261/2491] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:32 2022] DELETE /identity/v3/projects/fec9baec71844377959528f6027f88c1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:32.619681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.621153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.642455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:32.651790 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:32.668484 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8d3e2-22c0-4f1d-bc7d-ddf9e03d9d72 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.676264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7141cf9-a077-4137-9483-fd7e68b502aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.677220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7141cf9-a077-4137-9483-fd7e68b502aa admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.677511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7141cf9-a077-4137-9483-fd7e68b502aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.677823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7141cf9-a077-4137-9483-fd7e68b502aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.678560 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7141cf9-a077-4137-9483-fd7e68b502aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.681321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7141cf9-a077-4137-9483-fd7e68b502aa admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.696687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbe351-af8d-41f0-9960-f70f1ce9e6e0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.697072 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1231/2492] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:32.698478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406438e-661d-4910-9d98-bda49c51006e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.698888 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406438e-661d-4910-9d98-bda49c51006e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.699158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406438e-661d-4910-9d98-bda49c51006e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.701959 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.702941 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.703442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.703442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] PATH_INFO: `/v3/projects/02e545f0c2784698a35cd8dcb1b02395` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.704741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02e545f0c2784698a35cd8dcb1b02395)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.706703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.729334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9406438e-661d-4910-9d98-bda49c51006e None None] MFA Rules not processed for user `b1fa01f22f73426ea7910abb63ebd1aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:32.730692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9406438e-661d-4910-9d98-bda49c51006e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.761505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02e545f0c2784698a35cd8dcb1b02395', 'request_id': 'req-cf57cf13-8cb6-454a-bc62-69e317d3f288'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.767691 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02e545f0c2784698a35cd8dcb1b02395', 'request_id': 'req-cf57cf13-8cb6-454a-bc62-69e317d3f288'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.773223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02e545f0c2784698a35cd8dcb1b02395', 'request_id': 'req-cf57cf13-8cb6-454a-bc62-69e317d3f288'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.788225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] The token cache is being invalidate because project 02e545f0c2784698a35cd8dcb1b02395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:32.788544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.789350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.790950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1262/2493] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:32 2022] DELETE /identity/v3/projects/02e545f0c2784698a35cd8dcb1b02395 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:32.791809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.794307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.800882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1232/2494] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:32 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:32.801641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406438e-661d-4910-9d98-bda49c51006e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.803121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9406438e-661d-4910-9d98-bda49c51006e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.816025 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406438e-661d-4910-9d98-bda49c51006e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.863527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57cf13-8cb6-454a-bc62-69e317d3f288 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:32.868314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.868953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.869231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.869510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] PATH_INFO: `/v3/users/bf809bae7d7349348a1b9bd26743bdd3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.870764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf809bae7d7349348a1b9bd26743bdd3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.872683 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.873784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc783-da3b-4788-9c38-918f61cd02e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:32.874478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc783-da3b-4788-9c38-918f61cd02e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.874763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc783-da3b-4788-9c38-918f61cd02e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.875168 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc783-da3b-4788-9c38-918f61cd02e7 admin admin] PATH_INFO: `/v3/projects/0c63ce137e4c4ba6a168dc2bddae5b25/users/a31db786f09b4719a164010ff7086d11/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.892502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dc783-da3b-4788-9c38-918f61cd02e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c63ce137e4c4ba6a168dc2bddae5b25, user_id=a31db786f09b4719a164010ff7086d11, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:32.895359 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dc783-da3b-4788-9c38-918f61cd02e7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:32.910421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1233/2495] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:32 2022] PUT /identity/v3/projects/0c63ce137e4c4ba6a168dc2bddae5b25/users/a31db786f09b4719a164010ff7086d11/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:32.914789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76b302-f698-4d67-b927-7e7cbc435f98 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:32.915105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76b302-f698-4d67-b927-7e7cbc435f98 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:32.915387 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76b302-f698-4d67-b927-7e7cbc435f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:32.922312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf809bae7d7349348a1b9bd26743bdd3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.922967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf809bae7d7349348a1b9bd26743bdd3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.930659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:32.936427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7e76b302-f698-4d67-b927-7e7cbc435f98 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:32.936747 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf809bae7d7349348a1b9bd26743bdd3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.937643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e76b302-f698-4d67-b927-7e7cbc435f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.945723 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf809bae7d7349348a1b9bd26743bdd3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.955259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf809bae7d7349348a1b9bd26743bdd3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:32.961355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1263/2496] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:32 2022] DELETE /identity/v3/users/bf809bae7d7349348a1b9bd26743bdd3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:32.962231 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:32.963636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:32.983051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:33.007564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1234/2497] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:33.008431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76b302-f698-4d67-b927-7e7cbc435f98 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.010037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e76b302-f698-4d67-b927-7e7cbc435f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:33.023258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76b302-f698-4d67-b927-7e7cbc435f98 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.047293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce85b7c-65e6-46e5-bd0a-7a40223a7385 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.052089 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99e80cd-8c72-4b1a-9115-a1b47fb3b952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.052734 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e80cd-8c72-4b1a-9115-a1b47fb3b952 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.053307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e80cd-8c72-4b1a-9115-a1b47fb3b952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.053307 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99e80cd-8c72-4b1a-9115-a1b47fb3b952 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.053962 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e80cd-8c72-4b1a-9115-a1b47fb3b952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.055883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99e80cd-8c72-4b1a-9115-a1b47fb3b952 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.085871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1264/2498] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:32 2022] POST /identity/v3/projects => generated 405 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:33.087296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5b337-f64b-4c69-9d7a-d3d710b71599 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.087909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5b337-f64b-4c69-9d7a-d3d710b71599 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.088196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c5b337-f64b-4c69-9d7a-d3d710b71599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.096029 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf36eb1-504f-43e2-9e3b-058f759f1760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.096937 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf36eb1-504f-43e2-9e3b-058f759f1760 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.097424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf36eb1-504f-43e2-9e3b-058f759f1760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.097703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf36eb1-504f-43e2-9e3b-058f759f1760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.100766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf36eb1-504f-43e2-9e3b-058f759f1760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.102429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf36eb1-504f-43e2-9e3b-058f759f1760 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.113471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e4c5b337-f64b-4c69-9d7a-d3d710b71599 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:33.117039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e4c5b337-f64b-4c69-9d7a-d3d710b71599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:33.117166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1235/2499] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:33.118044 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf36eb1-504f-43e2-9e3b-058f759f1760 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.130930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf36eb1-504f-43e2-9e3b-058f759f1760 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.208775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1265/2500] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:33.210627 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5b337-f64b-4c69-9d7a-d3d710b71599 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.213013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6dc84a-9302-4136-928f-2c97f8a027af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.213809 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6dc84a-9302-4136-928f-2c97f8a027af admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.214191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6dc84a-9302-4136-928f-2c97f8a027af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.214556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6dc84a-9302-4136-928f-2c97f8a027af admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.215295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6dc84a-9302-4136-928f-2c97f8a027af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.217473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6dc84a-9302-4136-928f-2c97f8a027af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.220849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c5b337-f64b-4c69-9d7a-d3d710b71599 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.226067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c547741-dbca-4d48-b583-65e3a6c11e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.229075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c547741-dbca-4d48-b583-65e3a6c11e5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.229368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c547741-dbca-4d48-b583-65e3a6c11e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.229824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c547741-dbca-4d48-b583-65e3a6c11e5b admin admin] PATH_INFO: `/v3/projects/0c63ce137e4c4ba6a168dc2bddae5b25/users/a31db786f09b4719a164010ff7086d11/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.232240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c547741-dbca-4d48-b583-65e3a6c11e5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c63ce137e4c4ba6a168dc2bddae5b25, user_id=a31db786f09b4719a164010ff7086d11, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.235923 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c547741-dbca-4d48-b583-65e3a6c11e5b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.242613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1236/2501] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:33 2022] POST /identity/v3/users => generated 309 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:33.246500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6dc84a-9302-4136-928f-2c97f8a027af admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.251514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1266/2502] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:33 2022] PUT /identity/v3/projects/0c63ce137e4c4ba6a168dc2bddae5b25/users/a31db786f09b4719a164010ff7086d11/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:33.251514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c547741-dbca-4d48-b583-65e3a6c11e5b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.257881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6dc84a-9302-4136-928f-2c97f8a027af admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.343385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:33.345579 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb427c-c994-428f-81ff-eeb755a5aa39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.347235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb427c-c994-428f-81ff-eeb755a5aa39 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.347648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb427c-c994-428f-81ff-eeb755a5aa39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.348845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb427c-c994-428f-81ff-eeb755a5aa39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.351066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb427c-c994-428f-81ff-eeb755a5aa39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:33.353690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb427c-c994-428f-81ff-eeb755a5aa39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.356059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb427c-c994-428f-81ff-eeb755a5aa39 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.374454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1237/2503] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:00:33.377916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb427c-c994-428f-81ff-eeb755a5aa39 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.377916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-8feb427c-c994-428f-81ff-eeb755a5aa39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:33.382990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c547741-dbca-4d48-b583-65e3a6c11e5b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.387664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:33.388082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150567a-bd2b-4e8c-85e0-d83c441d4033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.389782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150567a-bd2b-4e8c-85e0-d83c441d4033 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.390266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150567a-bd2b-4e8c-85e0-d83c441d4033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.390564 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b150567a-bd2b-4e8c-85e0-d83c441d4033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.391470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150567a-bd2b-4e8c-85e0-d83c441d4033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.392831 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb427c-c994-428f-81ff-eeb755a5aa39 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.393275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b150567a-bd2b-4e8c-85e0-d83c441d4033 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.401360 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.402394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.402872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.403332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] PATH_INFO: `/v3/projects/8880e3b458c0405fb28ca3f9fd64ac37` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.404726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8880e3b458c0405fb28ca3f9fd64ac37)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.405244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1267/2504] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:33.406107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150567a-bd2b-4e8c-85e0-d83c441d4033 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.408429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.419297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b150567a-bd2b-4e8c-85e0-d83c441d4033 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.431878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dae9d-342d-4c13-aecb-a8791250facf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.432868 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dae9d-342d-4c13-aecb-a8791250facf admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.435429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dae9d-342d-4c13-aecb-a8791250facf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.436924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9dae9d-342d-4c13-aecb-a8791250facf admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.437595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9dae9d-342d-4c13-aecb-a8791250facf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.440744 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9dae9d-342d-4c13-aecb-a8791250facf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.483129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1268/2505] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:33 2022] POST /identity/v3/projects => generated 371 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:33.483129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dae9d-342d-4c13-aecb-a8791250facf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.495335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8880e3b458c0405fb28ca3f9fd64ac37', 'request_id': 'req-f88c0489-d3fa-4a66-8007-5032bec7e385'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:33.503460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8880e3b458c0405fb28ca3f9fd64ac37', 'request_id': 'req-f88c0489-d3fa-4a66-8007-5032bec7e385'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:33.515454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8880e3b458c0405fb28ca3f9fd64ac37', 'request_id': 'req-f88c0489-d3fa-4a66-8007-5032bec7e385'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:33.527434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] The token cache is being invalidate because project 8880e3b458c0405fb28ca3f9fd64ac37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:33.527802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:33.528524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:33.531281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1238/2506] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:33 2022] DELETE /identity/v3/projects/8880e3b458c0405fb28ca3f9fd64ac37 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:33.532227 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.536889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:33.562359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9dae9d-342d-4c13-aecb-a8791250facf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.567963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67951331-fa1f-4760-99a9-889aba2ae281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.568949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67951331-fa1f-4760-99a9-889aba2ae281 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.568949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67951331-fa1f-4760-99a9-889aba2ae281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.569196 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67951331-fa1f-4760-99a9-889aba2ae281 admin admin] PATH_INFO: `/v3/projects/97df054d46404e2986f0c4de9258f810/users/03658384e798469fb7bb990b25b29485/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.599330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67951331-fa1f-4760-99a9-889aba2ae281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97df054d46404e2986f0c4de9258f810, user_id=03658384e798469fb7bb990b25b29485, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.602511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67951331-fa1f-4760-99a9-889aba2ae281 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.625994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1269/2507] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:33 2022] PUT /identity/v3/projects/97df054d46404e2986f0c4de9258f810/users/03658384e798469fb7bb990b25b29485/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:33.629460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00b4a4-86a1-4019-9138-7e40c5344ad8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.629768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00b4a4-86a1-4019-9138-7e40c5344ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.630049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00b4a4-86a1-4019-9138-7e40c5344ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.630996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88c0489-d3fa-4a66-8007-5032bec7e385 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.676601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-aa00b4a4-86a1-4019-9138-7e40c5344ad8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:33.677863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa00b4a4-86a1-4019-9138-7e40c5344ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:33.741900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8816df5f-5063-46c4-af81-1d4755f0e4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.742542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816df5f-5063-46c4-af81-1d4755f0e4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.742827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816df5f-5063-46c4-af81-1d4755f0e4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.743080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8816df5f-5063-46c4-af81-1d4755f0e4b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.743752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816df5f-5063-46c4-af81-1d4755f0e4b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.746678 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8816df5f-5063-46c4-af81-1d4755f0e4b2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.755906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1270/2508] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:33.757346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00b4a4-86a1-4019-9138-7e40c5344ad8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.763902 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-aa00b4a4-86a1-4019-9138-7e40c5344ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:33.781006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa00b4a4-86a1-4019-9138-7e40c5344ad8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.783652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1239/2509] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:33 2022] POST /identity/v3/users => generated 286 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:33.786376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fb001-878a-46ef-94d0-ffbcc35e8190 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.786665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fb001-878a-46ef-94d0-ffbcc35e8190 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.786937 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fb001-878a-46ef-94d0-ffbcc35e8190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.790242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9efc3-8e2c-47e9-934e-c3ba9d42aa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.791031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9efc3-8e2c-47e9-934e-c3ba9d42aa95 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.791345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9efc3-8e2c-47e9-934e-c3ba9d42aa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.792366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e9efc3-8e2c-47e9-934e-c3ba9d42aa95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.792646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9efc3-8e2c-47e9-934e-c3ba9d42aa95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.795149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e9efc3-8e2c-47e9-934e-c3ba9d42aa95 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.809788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1271/2510] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:33.809788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9efc3-8e2c-47e9-934e-c3ba9d42aa95 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.826166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-500fb001-878a-46ef-94d0-ffbcc35e8190 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:33.826642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e9efc3-8e2c-47e9-934e-c3ba9d42aa95 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.827927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-500fb001-878a-46ef-94d0-ffbcc35e8190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:33.833288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3100dfd-7690-42a7-8f04-5c986cd5a5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.834186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3100dfd-7690-42a7-8f04-5c986cd5a5cd admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.834670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3100dfd-7690-42a7-8f04-5c986cd5a5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.835107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3100dfd-7690-42a7-8f04-5c986cd5a5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.835993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3100dfd-7690-42a7-8f04-5c986cd5a5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.838062 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3100dfd-7690-42a7-8f04-5c986cd5a5cd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.840096 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1240/2511] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:33.842180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fb001-878a-46ef-94d0-ffbcc35e8190 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.857400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1272/2512] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:33.859130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3100dfd-7690-42a7-8f04-5c986cd5a5cd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.860505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fb001-878a-46ef-94d0-ffbcc35e8190 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.865884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6db43-123c-488a-b2da-591ff2847fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.866684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6db43-123c-488a-b2da-591ff2847fea admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.867060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6db43-123c-488a-b2da-591ff2847fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.867429 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6db43-123c-488a-b2da-591ff2847fea admin admin] PATH_INFO: `/v3/projects/97df054d46404e2986f0c4de9258f810/users/03658384e798469fb7bb990b25b29485/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.869606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6db43-123c-488a-b2da-591ff2847fea admin admin] RBAC: Authorizing `identity:create_grant(project_id=97df054d46404e2986f0c4de9258f810, user_id=03658384e798469fb7bb990b25b29485, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.873413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3100dfd-7690-42a7-8f04-5c986cd5a5cd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.873866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6db43-123c-488a-b2da-591ff2847fea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.879912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdfe1b-db26-4c80-a435-45ae3c930360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.880556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdfe1b-db26-4c80-a435-45ae3c930360 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.880840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdfe1b-db26-4c80-a435-45ae3c930360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.881117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdfe1b-db26-4c80-a435-45ae3c930360 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.881858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdfe1b-db26-4c80-a435-45ae3c930360 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:33.882408 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdfe1b-db26-4c80-a435-45ae3c930360 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:33.884067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdfe1b-db26-4c80-a435-45ae3c930360 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:33.897830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1241/2513] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:33 2022] PUT /identity/v3/projects/97df054d46404e2986f0c4de9258f810/users/03658384e798469fb7bb990b25b29485/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:33.898789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6db43-123c-488a-b2da-591ff2847fea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.901158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1273/2514] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:00:33.902264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdfe1b-db26-4c80-a435-45ae3c930360 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:33.910491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:33.923038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdfe1b-db26-4c80-a435-45ae3c930360 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.929015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:33.990057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6db43-123c-488a-b2da-591ff2847fea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:33.996265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e3871-a525-4857-a372-74acafc5068c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:33.997132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e3871-a525-4857-a372-74acafc5068c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:33.997390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e3871-a525-4857-a372-74acafc5068c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:33.997615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e3871-a525-4857-a372-74acafc5068c admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:33.998829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e3871-a525-4857-a372-74acafc5068c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.001446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e3871-a525-4857-a372-74acafc5068c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.007258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986924c-44d3-4483-96a4-5bbd3e098ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:34.008674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986924c-44d3-4483-96a4-5bbd3e098ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.009295 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986924c-44d3-4483-96a4-5bbd3e098ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.009867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986924c-44d3-4483-96a4-5bbd3e098ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.012555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986924c-44d3-4483-96a4-5bbd3e098ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.014937 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986924c-44d3-4483-96a4-5bbd3e098ba8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.038128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1242/2515] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:33 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:34.038395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327cac-efb3-46be-89b6-96b6285ad9d7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.038930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327cac-efb3-46be-89b6-96b6285ad9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.039054 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92327cac-efb3-46be-89b6-96b6285ad9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.054202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1274/2516] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:33 2022] POST /identity/v3/projects => generated 365 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:34.055867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e3871-a525-4857-a372-74acafc5068c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:34.075677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-92327cac-efb3-46be-89b6-96b6285ad9d7 None None] MFA Rules not processed for user `03658384e798469fb7bb990b25b29485`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:34.076666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-92327cac-efb3-46be-89b6-96b6285ad9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:34.147337 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e3871-a525-4857-a372-74acafc5068c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:34.152966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d27b8f-f9dd-427d-a0d5-3e56c48351a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:34.153999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d27b8f-f9dd-427d-a0d5-3e56c48351a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.154573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d27b8f-f9dd-427d-a0d5-3e56c48351a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.155178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d27b8f-f9dd-427d-a0d5-3e56c48351a2 admin admin] PATH_INFO: `/v3/projects/d775913087b54deb9aefe7cd5179a01a/users/d0e0204de4304b3aaa7c867ea49628ea/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.161854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1243/2517] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:34 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:34.161854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92327cac-efb3-46be-89b6-96b6285ad9d7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:34.179349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d27b8f-f9dd-427d-a0d5-3e56c48351a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d775913087b54deb9aefe7cd5179a01a, user_id=d0e0204de4304b3aaa7c867ea49628ea, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.183571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d27b8f-f9dd-427d-a0d5-3e56c48351a2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.190662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92327cac-efb3-46be-89b6-96b6285ad9d7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:34.198861 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6332dd7-305b-46e3-9876-8f9b420697eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:34.199924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6332dd7-305b-46e3-9876-8f9b420697eb admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.200361 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6332dd7-305b-46e3-9876-8f9b420697eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.200791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6332dd7-305b-46e3-9876-8f9b420697eb admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.201771 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6332dd7-305b-46e3-9876-8f9b420697eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.202967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1275/2518] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:34 2022] PUT /identity/v3/projects/d775913087b54deb9aefe7cd5179a01a/users/d0e0204de4304b3aaa7c867ea49628ea/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:34.204046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d27b8f-f9dd-427d-a0d5-3e56c48351a2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:34.204944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6332dd7-305b-46e3-9876-8f9b420697eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.206397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f4d27b8f-f9dd-427d-a0d5-3e56c48351a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:34.238267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1244/2519] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:34 2022] POST /identity/v3/users => generated 283 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:34.240092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495f2a2-62d2-4c83-a809-3d3370acd533 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.240380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495f2a2-62d2-4c83-a809-3d3370acd533 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.240659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495f2a2-62d2-4c83-a809-3d3370acd533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.280102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d27b8f-f9dd-427d-a0d5-3e56c48351a2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:34.284103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bb473-ed36-4f1c-a4e2-5427058d3fc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:34.285066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bb473-ed36-4f1c-a4e2-5427058d3fc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.285502 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bb473-ed36-4f1c-a4e2-5427058d3fc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.285996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bb473-ed36-4f1c-a4e2-5427058d3fc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.290005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7495f2a2-62d2-4c83-a809-3d3370acd533 None None] MFA Rules not processed for user `d0e0204de4304b3aaa7c867ea49628ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:34.291473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7495f2a2-62d2-4c83-a809-3d3370acd533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:34.325383 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:34.367304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:34.376499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1245/2520] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:34.376707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bb473-ed36-4f1c-a4e2-5427058d3fc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.377954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495f2a2-62d2-4c83-a809-3d3370acd533 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:34.379300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5bb473-ed36-4f1c-a4e2-5427058d3fc9 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.394214 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1276/2521] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:34 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:34.395136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bb473-ed36-4f1c-a4e2-5427058d3fc9 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:34.462850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495f2a2-62d2-4c83-a809-3d3370acd533 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:34.469012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae2d51-f4e8-44e8-87f7-11388f495a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:34.469612 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae2d51-f4e8-44e8-87f7-11388f495a50 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.470758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae2d51-f4e8-44e8-87f7-11388f495a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.470758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ae2d51-f4e8-44e8-87f7-11388f495a50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.472083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae2d51-f4e8-44e8-87f7-11388f495a50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.473540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ae2d51-f4e8-44e8-87f7-11388f495a50 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.486301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1246/2522] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:34.490649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae2d51-f4e8-44e8-87f7-11388f495a50 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:34.506508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ae2d51-f4e8-44e8-87f7-11388f495a50 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:34.512932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e94e9-4acb-48b3-841a-07279f4f9024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:34.514333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e94e9-4acb-48b3-841a-07279f4f9024 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.514796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e94e9-4acb-48b3-841a-07279f4f9024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.515216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e94e9-4acb-48b3-841a-07279f4f9024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.516390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e94e9-4acb-48b3-841a-07279f4f9024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.517698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e94e9-4acb-48b3-841a-07279f4f9024 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.524002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5bb473-ed36-4f1c-a4e2-5427058d3fc9 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:34.535866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1247/2523] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:34.539650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e94e9-4acb-48b3-841a-07279f4f9024 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:34.540159 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88b0e4-71e2-4022-a23f-bd462fbda895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:34.540866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88b0e4-71e2-4022-a23f-bd462fbda895 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.541149 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88b0e4-71e2-4022-a23f-bd462fbda895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.541423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df88b0e4-71e2-4022-a23f-bd462fbda895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.542361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88b0e4-71e2-4022-a23f-bd462fbda895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.544940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df88b0e4-71e2-4022-a23f-bd462fbda895 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.561915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e94e9-4acb-48b3-841a-07279f4f9024 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:34.585116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0065f-bdbb-4f35-bd41-2b3bdc020c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:34.585913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0065f-bdbb-4f35-bd41-2b3bdc020c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.586195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0065f-bdbb-4f35-bd41-2b3bdc020c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.586470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe0065f-bdbb-4f35-bd41-2b3bdc020c41 admin admin] PATH_INFO: `/v3/projects/ebed0772fd2548d3a7c6725944c8d634/users/97b50576f8ef4fe4b67cc631737b8d0b/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.593144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1277/2524] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:34 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:34.594247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88b0e4-71e2-4022-a23f-bd462fbda895 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:34.609997 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0065f-bdbb-4f35-bd41-2b3bdc020c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebed0772fd2548d3a7c6725944c8d634, user_id=97b50576f8ef4fe4b67cc631737b8d0b, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.612804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe0065f-bdbb-4f35-bd41-2b3bdc020c41 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.642518 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1248/2525] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:34 2022] PUT /identity/v3/projects/ebed0772fd2548d3a7c6725944c8d634/users/97b50576f8ef4fe4b67cc631737b8d0b/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:00:34.643586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0065f-bdbb-4f35-bd41-2b3bdc020c41 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:34.681446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df88b0e4-71e2-4022-a23f-bd462fbda895 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:34.742126 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe0065f-bdbb-4f35-bd41-2b3bdc020c41 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:34.747632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de39bad-e94a-41fc-aa92-c7aa798de46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:34.748319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de39bad-e94a-41fc-aa92-c7aa798de46d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.748597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de39bad-e94a-41fc-aa92-c7aa798de46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.748871 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de39bad-e94a-41fc-aa92-c7aa798de46d admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.749545 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de39bad-e94a-41fc-aa92-c7aa798de46d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.751704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de39bad-e94a-41fc-aa92-c7aa798de46d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.768951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:34.769944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.770303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.770620 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] PATH_INFO: `/v3/users/b1fa01f22f73426ea7910abb63ebd1aa` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.771936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1fa01f22f73426ea7910abb63ebd1aa)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:34.774869 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:34.777519 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:34.791010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1249/2526] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:34 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:34.792389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5513d5f-0ccb-4c6a-b0b4-809042629fac None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.792675 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5513d5f-0ccb-4c6a-b0b4-809042629fac None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.792946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5513d5f-0ccb-4c6a-b0b4-809042629fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.832467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5513d5f-0ccb-4c6a-b0b4-809042629fac None None] MFA Rules not processed for user `97b50576f8ef4fe4b67cc631737b8d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:34.834412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5513d5f-0ccb-4c6a-b0b4-809042629fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:34.850934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1fa01f22f73426ea7910abb63ebd1aa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:34.852044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1fa01f22f73426ea7910abb63ebd1aa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:34.866979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1fa01f22f73426ea7910abb63ebd1aa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:34.877343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1fa01f22f73426ea7910abb63ebd1aa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:34.886765 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:34.891441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1fa01f22f73426ea7910abb63ebd1aa'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:34.904797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1278/2527] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:34 2022] DELETE /identity/v3/users/b1fa01f22f73426ea7910abb63ebd1aa => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:34.906315 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:34.909672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:34.926177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1250/2528] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:34.927773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f1ae2-0dc7-48c1-9099-cb1f3b257578 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:34.928252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f1ae2-0dc7-48c1-9099-cb1f3b257578 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:34.928766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f1ae2-0dc7-48c1-9099-cb1f3b257578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:34.952321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a37f1ae2-0dc7-48c1-9099-cb1f3b257578 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:34.953716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a37f1ae2-0dc7-48c1-9099-cb1f3b257578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:35.020236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c642c0-e1d8-4667-8dd0-fb0f3bad50e7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.029659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8b910-89a8-4822-9284-986106bca373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:35.030537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8b910-89a8-4822-9284-986106bca373 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.031091 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8b910-89a8-4822-9284-986106bca373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.031473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f8b910-89a8-4822-9284-986106bca373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.032345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8b910-89a8-4822-9284-986106bca373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:35.034888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f8b910-89a8-4822-9284-986106bca373 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:35.042837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1251/2529] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:35.042837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f1ae2-0dc7-48c1-9099-cb1f3b257578 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:35.043712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a37f1ae2-0dc7-48c1-9099-cb1f3b257578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:35.051433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1279/2530] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:34 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:35.052554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8b910-89a8-4822-9284-986106bca373 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:35.054575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-16f8b910-89a8-4822-9284-986106bca373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:35.061618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f1ae2-0dc7-48c1-9099-cb1f3b257578 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.070203 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12985903-3a2b-4649-af15-44184fcd5d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:35.074652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12985903-3a2b-4649-af15-44184fcd5d47 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.075550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12985903-3a2b-4649-af15-44184fcd5d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.075890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12985903-3a2b-4649-af15-44184fcd5d47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.076676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12985903-3a2b-4649-af15-44184fcd5d47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:35.083808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12985903-3a2b-4649-af15-44184fcd5d47 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:35.116343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f8b910-89a8-4822-9284-986106bca373 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.120681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b6575-b4a4-482b-a724-643595ba9bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:35.121376 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b6575-b4a4-482b-a724-643595ba9bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.121826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b6575-b4a4-482b-a724-643595ba9bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.122154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b6575-b4a4-482b-a724-643595ba9bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.131496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1252/2531] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:35 2022] POST /identity/v3/projects => generated 361 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:35.132191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12985903-3a2b-4649-af15-44184fcd5d47 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:35.135058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b6575-b4a4-482b-a724-643595ba9bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:35.136934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b6575-b4a4-482b-a724-643595ba9bb1 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:35.149638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1280/2532] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:35.152333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b6575-b4a4-482b-a724-643595ba9bb1 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:35.166139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b6575-b4a4-482b-a724-643595ba9bb1 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.186403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:35.212058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12985903-3a2b-4649-af15-44184fcd5d47 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.218889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822167c-a82c-48a8-b83d-17e82d3c9c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:35.219790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822167c-a82c-48a8-b83d-17e82d3c9c83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.220135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822167c-a82c-48a8-b83d-17e82d3c9c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.220500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822167c-a82c-48a8-b83d-17e82d3c9c83 admin admin] PATH_INFO: `/v3/projects/764f8182f42341f29ba66b31f5c7b29c/users/a36dfedd99064b08abbaf3ddc4b10ba2/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.238188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822167c-a82c-48a8-b83d-17e82d3c9c83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=764f8182f42341f29ba66b31f5c7b29c, user_id=a36dfedd99064b08abbaf3ddc4b10ba2, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:35.241168 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822167c-a82c-48a8-b83d-17e82d3c9c83 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:35.263110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1253/2533] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:35 2022] PUT /identity/v3/projects/764f8182f42341f29ba66b31f5c7b29c/users/a36dfedd99064b08abbaf3ddc4b10ba2/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:35.263592 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2666b-c5ff-4696-8712-3e71d12488f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:35.264405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2666b-c5ff-4696-8712-3e71d12488f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.264871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2666b-c5ff-4696-8712-3e71d12488f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.265156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2666b-c5ff-4696-8712-3e71d12488f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.265861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2666b-c5ff-4696-8712-3e71d12488f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:35.268225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c2666b-c5ff-4696-8712-3e71d12488f0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:35.269832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822167c-a82c-48a8-b83d-17e82d3c9c83 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:35.315134 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1281/2534] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:35 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:35.321947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2666b-c5ff-4696-8712-3e71d12488f0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:35.344369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:35.347960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2666b-c5ff-4696-8712-3e71d12488f0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.393909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822167c-a82c-48a8-b83d-17e82d3c9c83 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.400376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b6ec9-bcbd-48e1-b7ac-19c32940ecfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:35.401178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b6ec9-bcbd-48e1-b7ac-19c32940ecfe admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.401565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b6ec9-bcbd-48e1-b7ac-19c32940ecfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.402021 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b6ec9-bcbd-48e1-b7ac-19c32940ecfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.402946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b6ec9-bcbd-48e1-b7ac-19c32940ecfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:35.405599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36b6ec9-bcbd-48e1-b7ac-19c32940ecfe admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:35.422406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1254/2535] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:35.429374 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b6ec9-bcbd-48e1-b7ac-19c32940ecfe admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:35.451236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36b6ec9-bcbd-48e1-b7ac-19c32940ecfe admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.457786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403dd9ea-df57-4793-b598-7212507bb2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:35.458472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dd9ea-df57-4793-b598-7212507bb2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.458759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dd9ea-df57-4793-b598-7212507bb2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.459039 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403dd9ea-df57-4793-b598-7212507bb2bb admin admin] PATH_INFO: `/v3/projects/764f8182f42341f29ba66b31f5c7b29c/users/a36dfedd99064b08abbaf3ddc4b10ba2/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.461430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403dd9ea-df57-4793-b598-7212507bb2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=764f8182f42341f29ba66b31f5c7b29c, user_id=a36dfedd99064b08abbaf3ddc4b10ba2, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:35.471333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403dd9ea-df57-4793-b598-7212507bb2bb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:35.486020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0cfe-9b40-4244-a506-a9c8df5ee328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:35.487881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0cfe-9b40-4244-a506-a9c8df5ee328 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.488747 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0cfe-9b40-4244-a506-a9c8df5ee328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.489218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e0cfe-9b40-4244-a506-a9c8df5ee328 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.491233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e0cfe-9b40-4244-a506-a9c8df5ee328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:35.495166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e0cfe-9b40-4244-a506-a9c8df5ee328 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:35.495316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1255/2536] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:35 2022] PUT /identity/v3/projects/764f8182f42341f29ba66b31f5c7b29c/users/a36dfedd99064b08abbaf3ddc4b10ba2/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:35.496555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dac6c-1b68-42db-8b2e-689049d86c6d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.496986 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dac6c-1b68-42db-8b2e-689049d86c6d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.497335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4dac6c-1b68-42db-8b2e-689049d86c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.519584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1282/2537] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:35.523916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0cfe-9b40-4244-a506-a9c8df5ee328 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:35.537142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e0cfe-9b40-4244-a506-a9c8df5ee328 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.559890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9d4dac6c-1b68-42db-8b2e-689049d86c6d None None] MFA Rules not processed for user `a36dfedd99064b08abbaf3ddc4b10ba2`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:35.561550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9d4dac6c-1b68-42db-8b2e-689049d86c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:35.664719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c709-6f7c-435c-8c52-634553f43873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:35.668140 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c709-6f7c-435c-8c52-634553f43873 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.669598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c709-6f7c-435c-8c52-634553f43873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.670669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e361c709-6f7c-435c-8c52-634553f43873 admin admin] PATH_INFO: `/v3/projects/0b4ab54f3c854671a9b0e8403f902ff0/users/8914f965db7641679c1c8cc872803c09/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.708650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361c709-6f7c-435c-8c52-634553f43873 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4ab54f3c854671a9b0e8403f902ff0, user_id=8914f965db7641679c1c8cc872803c09, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:35.717499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e361c709-6f7c-435c-8c52-634553f43873 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:35.759027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1283/2538] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:35 2022] PUT /identity/v3/projects/0b4ab54f3c854671a9b0e8403f902ff0/users/8914f965db7641679c1c8cc872803c09/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:35.774681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c709-6f7c-435c-8c52-634553f43873 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:35.857030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1256/2539] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:35 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 362 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:35.859009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4dac6c-1b68-42db-8b2e-689049d86c6d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:35.864938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9d4dac6c-1b68-42db-8b2e-689049d86c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:35.902544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4dac6c-1b68-42db-8b2e-689049d86c6d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.903078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:35.961127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e361c709-6f7c-435c-8c52-634553f43873 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:35.974336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b51ae-9dd8-400a-91a4-c370f7632bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:35.975249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b51ae-9dd8-400a-91a4-c370f7632bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:35.975603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b51ae-9dd8-400a-91a4-c370f7632bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:35.975954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b51ae-9dd8-400a-91a4-c370f7632bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:35.976743 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b51ae-9dd8-400a-91a4-c370f7632bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:35.978698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30b51ae-9dd8-400a-91a4-c370f7632bb5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:35.998969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1284/2540] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:35 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:35.998969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b51ae-9dd8-400a-91a4-c370f7632bb5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:36.006331 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f30b51ae-9dd8-400a-91a4-c370f7632bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:36.098242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:36.099121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:36.101174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:36.101571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] PATH_INFO: `/v3/projects/2ca9bdb96092498fb692fb58deab254a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:36.105554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ca9bdb96092498fb692fb58deab254a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:36.110574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:36.189698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b51ae-9dd8-400a-91a4-c370f7632bb5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:36.194375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d20c28-04da-4e5b-b3a2-24c0218330c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:36.195280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20c28-04da-4e5b-b3a2-24c0218330c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:36.195674 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20c28-04da-4e5b-b3a2-24c0218330c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:36.196385 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d20c28-04da-4e5b-b3a2-24c0218330c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:36.232658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ca9bdb96092498fb692fb58deab254a', 'request_id': 'req-ef231196-0d30-4413-99e8-ec13d3a9e00f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:36.234666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:36.244948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ca9bdb96092498fb692fb58deab254a', 'request_id': 'req-ef231196-0d30-4413-99e8-ec13d3a9e00f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:36.258755 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ca9bdb96092498fb692fb58deab254a', 'request_id': 'req-ef231196-0d30-4413-99e8-ec13d3a9e00f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:36.282610 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] The token cache is being invalidate because project 2ca9bdb96092498fb692fb58deab254a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:36.283058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:36.283868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:36.286846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1257/2541] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:35 2022] DELETE /identity/v3/projects/2ca9bdb96092498fb692fb58deab254a => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:36.288421 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:36.290173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:36.315246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d20c28-04da-4e5b-b3a2-24c0218330c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:36.317327 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d20c28-04da-4e5b-b3a2-24c0218330c9 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:36.334528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1285/2542] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:35 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:36.335750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d20c28-04da-4e5b-b3a2-24c0218330c9 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:36.351425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d20c28-04da-4e5b-b3a2-24c0218330c9 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:36.377665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef231196-0d30-4413-99e8-ec13d3a9e00f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:36.391438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633a002-0722-4ae4-bbda-2026dd9b85fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:36.392324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a002-0722-4ae4-bbda-2026dd9b85fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:36.392742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a002-0722-4ae4-bbda-2026dd9b85fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:36.392927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f633a002-0722-4ae4-bbda-2026dd9b85fd admin admin] PATH_INFO: `/v3/projects/0b4ab54f3c854671a9b0e8403f902ff0/users/8914f965db7641679c1c8cc872803c09/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:36.400867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633a002-0722-4ae4-bbda-2026dd9b85fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b4ab54f3c854671a9b0e8403f902ff0, user_id=8914f965db7641679c1c8cc872803c09, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:36.404901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f633a002-0722-4ae4-bbda-2026dd9b85fd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:36.431425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1258/2543] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:36 2022] PUT /identity/v3/projects/0b4ab54f3c854671a9b0e8403f902ff0/users/8914f965db7641679c1c8cc872803c09/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:36.488438 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:36.489654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:36.490310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633a002-0722-4ae4-bbda-2026dd9b85fd admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:36.491739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:36.492145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f633a002-0722-4ae4-bbda-2026dd9b85fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:36.492498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] PATH_INFO: `/v3/users/03658384e798469fb7bb990b25b29485` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:36.494249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] RBAC: Authorizing `identity:delete_user(user_id=03658384e798469fb7bb990b25b29485)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:36.497023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:36.520447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:36.573114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03658384e798469fb7bb990b25b29485'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:36.574343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03658384e798469fb7bb990b25b29485'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:36.583308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f633a002-0722-4ae4-bbda-2026dd9b85fd admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:36.589826 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03658384e798469fb7bb990b25b29485'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:36.590709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb1547-551f-4d7a-99bd-275b22ae2407 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:36.591405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb1547-551f-4d7a-99bd-275b22ae2407 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:36.591687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb1547-551f-4d7a-99bd-275b22ae2407 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:36.592163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb1547-551f-4d7a-99bd-275b22ae2407 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:36.594355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60bb1547-551f-4d7a-99bd-275b22ae2407 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:36.601000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03658384e798469fb7bb990b25b29485'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:36.612995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-e8542f8b-10a5-45e2-9e3b-a8226f1fa98d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03658384e798469fb7bb990b25b29485'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:36.624232 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1286/2544] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:36 2022] DELETE /identity/v3/users/03658384e798469fb7bb990b25b29485 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:36.628304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96639-441c-4b31-890c-7b50e02e1d3f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:36.628808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96639-441c-4b31-890c-7b50e02e1d3f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:36.630144 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc96639-441c-4b31-890c-7b50e02e1d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:36.679391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9dc96639-441c-4b31-890c-7b50e02e1d3f None None] MFA Rules not processed for user `8914f965db7641679c1c8cc872803c09`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:36.681101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9dc96639-441c-4b31-890c-7b50e02e1d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:36.689638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb1547-551f-4d7a-99bd-275b22ae2407 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:36.696136 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bb1547-551f-4d7a-99bd-275b22ae2407 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:36.723199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1259/2545] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:36.964576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1287/2546] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:36 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:36.974290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb1547-551f-4d7a-99bd-275b22ae2407 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:36.975627 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-60bb1547-551f-4d7a-99bd-275b22ae2407 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:37.013511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb1547-551f-4d7a-99bd-275b22ae2407 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:37.109075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:37.155672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19ec69-72b8-4cf4-8b4c-d3cab04b4467 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:37.156650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19ec69-72b8-4cf4-8b4c-d3cab04b4467 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:37.157038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19ec69-72b8-4cf4-8b4c-d3cab04b4467 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:37.157218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19ec69-72b8-4cf4-8b4c-d3cab04b4467 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:37.167655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19ec69-72b8-4cf4-8b4c-d3cab04b4467 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:37.169310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19ec69-72b8-4cf4-8b4c-d3cab04b4467 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:37.206142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1260/2547] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:36 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:37.311507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc96639-441c-4b31-890c-7b50e02e1d3f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:37.313324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19ec69-72b8-4cf4-8b4c-d3cab04b4467 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:37.316092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4e19ec69-72b8-4cf4-8b4c-d3cab04b4467 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:37.317524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9dc96639-441c-4b31-890c-7b50e02e1d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:37.351293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19ec69-72b8-4cf4-8b4c-d3cab04b4467 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:37.360883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0213d5-2df0-4045-a5df-25540b7e1f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:37.363465 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0213d5-2df0-4045-a5df-25540b7e1f8e service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:37.363971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0213d5-2df0-4045-a5df-25540b7e1f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:37.364424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0213d5-2df0-4045-a5df-25540b7e1f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:37.379479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d0213d5-2df0-4045-a5df-25540b7e1f8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:37.446296 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc96639-441c-4b31-890c-7b50e02e1d3f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:37.461416 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:37.462746 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:37.463219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:37.464398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] PATH_INFO: `/v3/projects/97df054d46404e2986f0c4de9258f810` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:37.465985 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97df054d46404e2986f0c4de9258f810)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:37.468446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:37.497819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:37.540271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97df054d46404e2986f0c4de9258f810', 'request_id': 'req-80f75bbf-f800-4c32-82cb-c78e96f9a299'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:37.545345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0213d5-2df0-4045-a5df-25540b7e1f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:37.546857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0213d5-2df0-4045-a5df-25540b7e1f8e service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:37.555546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97df054d46404e2986f0c4de9258f810', 'request_id': 'req-80f75bbf-f800-4c32-82cb-c78e96f9a299'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:37.563084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97df054d46404e2986f0c4de9258f810', 'request_id': 'req-80f75bbf-f800-4c32-82cb-c78e96f9a299'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:37.574324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1261/2548] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:37.607640 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] The token cache is being invalidate because project 97df054d46404e2986f0c4de9258f810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:37.608679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:37.612763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-80f75bbf-f800-4c32-82cb-c78e96f9a299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:37.621611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1288/2549] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:37 2022] DELETE /identity/v3/projects/97df054d46404e2986f0c4de9258f810 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:37.622906 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1928ca-55e5-44c5-9845-5653e0b03eea None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:37.623315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1928ca-55e5-44c5-9845-5653e0b03eea None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:37.623600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1928ca-55e5-44c5-9845-5653e0b03eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:37.665359 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6b1928ca-55e5-44c5-9845-5653e0b03eea None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:37.665638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6b1928ca-55e5-44c5-9845-5653e0b03eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:37.833571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1262/2550] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:37.837792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62411d6e-f4a9-4244-8be4-24c80f8281d0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:37.838109 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62411d6e-f4a9-4244-8be4-24c80f8281d0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:37.838367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62411d6e-f4a9-4244-8be4-24c80f8281d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:37.897170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-62411d6e-f4a9-4244-8be4-24c80f8281d0 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:37.899158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-62411d6e-f4a9-4244-8be4-24c80f8281d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:37.920489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1289/2551] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:37.921612 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1928ca-55e5-44c5-9845-5653e0b03eea None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:37.940083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1928ca-55e5-44c5-9845-5653e0b03eea None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:37.952230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d02a26-7139-4093-b1ce-bfe3e64406f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:37.953064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d02a26-7139-4093-b1ce-bfe3e64406f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:37.953354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d02a26-7139-4093-b1ce-bfe3e64406f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:37.953632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d02a26-7139-4093-b1ce-bfe3e64406f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:37.954616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d02a26-7139-4093-b1ce-bfe3e64406f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:37.954769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d02a26-7139-4093-b1ce-bfe3e64406f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:37.956350 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d02a26-7139-4093-b1ce-bfe3e64406f3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:37.986806 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1263/2552] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:00:37.991656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62411d6e-f4a9-4244-8be4-24c80f8281d0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:38.003573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62411d6e-f4a9-4244-8be4-24c80f8281d0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:38.008952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bd400-d58d-4b2f-9d4c-ddf651f6e2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:38.009836 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bd400-d58d-4b2f-9d4c-ddf651f6e2e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:38.010228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bd400-d58d-4b2f-9d4c-ddf651f6e2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:38.010585 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5bd400-d58d-4b2f-9d4c-ddf651f6e2e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:38.011783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5bd400-d58d-4b2f-9d4c-ddf651f6e2e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:38.014330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5bd400-d58d-4b2f-9d4c-ddf651f6e2e6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:38.078724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1290/2553] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:37 2022] POST /identity/v3/projects => generated 403 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:38.083670 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d02a26-7139-4093-b1ce-bfe3e64406f3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:38.102149 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d02a26-7139-4093-b1ce-bfe3e64406f3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:38.213447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:38.226840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a8148-e87c-42c1-9ce2-9831b932c291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:38.228181 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a8148-e87c-42c1-9ce2-9831b932c291 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:38.228706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a8148-e87c-42c1-9ce2-9831b932c291 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:38.229941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7a8148-e87c-42c1-9ce2-9831b932c291 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:38.231151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a8148-e87c-42c1-9ce2-9831b932c291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:38.242009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7a8148-e87c-42c1-9ce2-9831b932c291 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:38.291322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1264/2554] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:38 2022] POST /identity/v3/users => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:38.296177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bd400-d58d-4b2f-9d4c-ddf651f6e2e6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:38.317587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5bd400-d58d-4b2f-9d4c-ddf651f6e2e6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:38.335830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fc1e3-16d1-474d-970b-e7b4c151d7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:38.337028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fc1e3-16d1-474d-970b-e7b4c151d7c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:38.337318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fc1e3-16d1-474d-970b-e7b4c151d7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:38.337593 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fc1e3-16d1-474d-970b-e7b4c151d7c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:38.338346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fc1e3-16d1-474d-970b-e7b4c151d7c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:38.341030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fc1e3-16d1-474d-970b-e7b4c151d7c8 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:38.387799 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a8148-e87c-42c1-9ce2-9831b932c291 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:38.403552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1291/2555] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:38.410671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7a8148-e87c-42c1-9ce2-9831b932c291 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:38.420437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d0381-029d-4977-bac8-f54dc220db1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:38.422397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d0381-029d-4977-bac8-f54dc220db1a admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:38.431535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d0381-029d-4977-bac8-f54dc220db1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:38.432045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d0381-029d-4977-bac8-f54dc220db1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:38.432838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d0381-029d-4977-bac8-f54dc220db1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:38.434921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d0381-029d-4977-bac8-f54dc220db1a admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:38.462871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fc1e3-16d1-474d-970b-e7b4c151d7c8 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:38.471379 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1265/2556] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:00:38.477365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fc1e3-16d1-474d-970b-e7b4c151d7c8 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:38.496786 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfc2f0-b44e-47e4-aae4-784967e0a9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:38.499518 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfc2f0-b44e-47e4-aae4-784967e0a9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:38.500107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfc2f0-b44e-47e4-aae4-784967e0a9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:38.500214 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cfc2f0-b44e-47e4-aae4-784967e0a9bf admin admin] PATH_INFO: `/v3/projects/19287120410743e6a7bdc3347c778385/users/753024d13b614c6ca50942d8b481e79f/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:38.540891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cfc2f0-b44e-47e4-aae4-784967e0a9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=19287120410743e6a7bdc3347c778385, user_id=753024d13b614c6ca50942d8b481e79f, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:38.549267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cfc2f0-b44e-47e4-aae4-784967e0a9bf admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:38.594654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1292/2557] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:38 2022] PUT /identity/v3/projects/19287120410743e6a7bdc3347c778385/users/753024d13b614c6ca50942d8b481e79f/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:38.595988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f27ae9-5456-4225-b1f3-c90ed24a3c84 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:38.596386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f27ae9-5456-4225-b1f3-c90ed24a3c84 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:38.596816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f27ae9-5456-4225-b1f3-c90ed24a3c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:38.665643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d0f27ae9-5456-4225-b1f3-c90ed24a3c84 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:38.667636 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d0f27ae9-5456-4225-b1f3-c90ed24a3c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:38.805325 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1266/2558] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:38.818888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfc2f0-b44e-47e4-aae4-784967e0a9bf admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:38.824881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-17cfc2f0-b44e-47e4-aae4-784967e0a9bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:38.945083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cfc2f0-b44e-47e4-aae4-784967e0a9bf admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:38.951920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8997fe0-cbd0-48cc-aef1-1055c0775ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:38.953191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8997fe0-cbd0-48cc-aef1-1055c0775ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:38.953191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8997fe0-cbd0-48cc-aef1-1055c0775ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:38.953191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8997fe0-cbd0-48cc-aef1-1055c0775ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:38.981019 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8997fe0-cbd0-48cc-aef1-1055c0775ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:38.982649 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8997fe0-cbd0-48cc-aef1-1055c0775ef2 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:38.996762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1293/2559] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:00:39.683752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf027e-50ef-4225-a47a-6bde294bdc4f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:39.684420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf027e-50ef-4225-a47a-6bde294bdc4f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:39.684706 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf027e-50ef-4225-a47a-6bde294bdc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:39.732941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-69bf027e-50ef-4225-a47a-6bde294bdc4f None None] MFA Rules not processed for user `a31db786f09b4719a164010ff7086d11`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:39.735709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69bf027e-50ef-4225-a47a-6bde294bdc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:39.867676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1267/2560] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:39.880544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8997fe0-cbd0-48cc-aef1-1055c0775ef2 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:39.882164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b8997fe0-cbd0-48cc-aef1-1055c0775ef2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:39.956400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:39.964584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8997fe0-cbd0-48cc-aef1-1055c0775ef2 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:39.976078 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e48830-afe0-4f3a-b533-22558d303ded service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c6943686c843e588df14f7c6dffb8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:39.978198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e48830-afe0-4f3a-b533-22558d303ded service glance] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:39.978884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e48830-afe0-4f3a-b533-22558d303ded service glance] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:39.979594 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e48830-afe0-4f3a-b533-22558d303ded service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:39.983906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-23e48830-afe0-4f3a-b533-22558d303ded service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:40.047680 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a74329-b9ad-4fa8-b33a-d244044d1a52 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:40.048071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a74329-b9ad-4fa8-b33a-d244044d1a52 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:40.048552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a74329-b9ad-4fa8-b33a-d244044d1a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:40.078377 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e48830-afe0-4f3a-b533-22558d303ded service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:40.080491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e48830-afe0-4f3a-b533-22558d303ded service glance] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:40.082822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a1a74329-b9ad-4fa8-b33a-d244044d1a52 None None] MFA Rules not processed for user `753024d13b614c6ca50942d8b481e79f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:40.084043 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a1a74329-b9ad-4fa8-b33a-d244044d1a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:40.095608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1294/2561] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:40.154713 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:40.181520 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1268/2562] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:40.185988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e48830-afe0-4f3a-b533-22558d303ded service glance] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:40.219558 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e48830-afe0-4f3a-b533-22558d303ded service glance] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:40.230570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b6384-d11a-4359-b332-bfe3d4f2b770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:40.231595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b6384-d11a-4359-b332-bfe3d4f2b770 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:40.232804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b6384-d11a-4359-b332-bfe3d4f2b770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:40.233334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b6384-d11a-4359-b332-bfe3d4f2b770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:40.235700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b6384-d11a-4359-b332-bfe3d4f2b770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:40.239032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476b6384-d11a-4359-b332-bfe3d4f2b770 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:40.279209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcfb6e-3006-4f85-bb68-147ac0ba5f08 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:40.279796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcfb6e-3006-4f85-bb68-147ac0ba5f08 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:40.280237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fcfb6e-3006-4f85-bb68-147ac0ba5f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:40.306951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1295/2563] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:40 2022] POST /identity/v3/projects => generated 403 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:00:40.309765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b6384-d11a-4359-b332-bfe3d4f2b770 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:40.364704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-85fcfb6e-3006-4f85-bb68-147ac0ba5f08 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:40.366606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-85fcfb6e-3006-4f85-bb68-147ac0ba5f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:40.439676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b6384-d11a-4359-b332-bfe3d4f2b770 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:40.446290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d019a9-cea5-406c-8ecb-31fcbc47dc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:40.446589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d019a9-cea5-406c-8ecb-31fcbc47dc7f admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:40.446870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d019a9-cea5-406c-8ecb-31fcbc47dc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:40.447143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d019a9-cea5-406c-8ecb-31fcbc47dc7f admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:40.448171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d019a9-cea5-406c-8ecb-31fcbc47dc7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:40.449767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d019a9-cea5-406c-8ecb-31fcbc47dc7f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:40.469268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1269/2564] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 08:00:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:40.473025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1296/2565] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:40 2022] POST /identity/v3/users => generated 308 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:40.494992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcfb6e-3006-4f85-bb68-147ac0ba5f08 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:40.508737 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fcfb6e-3006-4f85-bb68-147ac0ba5f08 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:40.513996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66773c70-4d34-4430-9f8e-3985bcedb76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:40.514741 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66773c70-4d34-4430-9f8e-3985bcedb76b admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:40.515088 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66773c70-4d34-4430-9f8e-3985bcedb76b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:40.515430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66773c70-4d34-4430-9f8e-3985bcedb76b admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:40.516210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66773c70-4d34-4430-9f8e-3985bcedb76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:40.518064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66773c70-4d34-4430-9f8e-3985bcedb76b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:40.526238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d019a9-cea5-406c-8ecb-31fcbc47dc7f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:40.527996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-42d019a9-cea5-406c-8ecb-31fcbc47dc7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:40.529770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1270/2566] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:40.534537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66773c70-4d34-4430-9f8e-3985bcedb76b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:40.545188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66773c70-4d34-4430-9f8e-3985bcedb76b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:40.549996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c37095-05f3-49af-9728-ca921ae7218b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:40.550683 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c37095-05f3-49af-9728-ca921ae7218b admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:40.550966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c37095-05f3-49af-9728-ca921ae7218b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:40.551244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c37095-05f3-49af-9728-ca921ae7218b admin admin] PATH_INFO: `/v3/projects/99c41edea3da461dbf21dd5ae8085d22/users/594b1ef30080423a89ff8bbc96da881f/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:40.571745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c37095-05f3-49af-9728-ca921ae7218b admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c41edea3da461dbf21dd5ae8085d22, user_id=594b1ef30080423a89ff8bbc96da881f, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:40.574693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c37095-05f3-49af-9728-ca921ae7218b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:40.592603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1271/2567] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:40 2022] PUT /identity/v3/projects/99c41edea3da461dbf21dd5ae8085d22/users/594b1ef30080423a89ff8bbc96da881f/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:40.593556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c37095-05f3-49af-9728-ca921ae7218b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:40.614350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:40.643248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d019a9-cea5-406c-8ecb-31fcbc47dc7f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:40.666685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c37095-05f3-49af-9728-ca921ae7218b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:40.671788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30b100-5472-48cc-8a2c-a93944d66967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:40.672427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30b100-5472-48cc-8a2c-a93944d66967 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:40.672707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30b100-5472-48cc-8a2c-a93944d66967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:40.672979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30b100-5472-48cc-8a2c-a93944d66967 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:40.673679 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30b100-5472-48cc-8a2c-a93944d66967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:40.675321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30b100-5472-48cc-8a2c-a93944d66967 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:40.685780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1272/2568] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:40.689876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30b100-5472-48cc-8a2c-a93944d66967 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:40.704056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30b100-5472-48cc-8a2c-a93944d66967 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:40.712401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d923d-6676-4a65-a793-3c469d970ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:40.713102 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d923d-6676-4a65-a793-3c469d970ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:40.713388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d923d-6676-4a65-a793-3c469d970ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:40.713667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942d923d-6676-4a65-a793-3c469d970ef0 admin admin] PATH_INFO: `/v3/projects/99c41edea3da461dbf21dd5ae8085d22/users/594b1ef30080423a89ff8bbc96da881f/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:40.718616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d923d-6676-4a65-a793-3c469d970ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99c41edea3da461dbf21dd5ae8085d22, user_id=594b1ef30080423a89ff8bbc96da881f, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:40.722432 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942d923d-6676-4a65-a793-3c469d970ef0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:40.728550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35628bc6-a395-4827-af4c-faaedf182408 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:40.729242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35628bc6-a395-4827-af4c-faaedf182408 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:40.729610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35628bc6-a395-4827-af4c-faaedf182408 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:40.729947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35628bc6-a395-4827-af4c-faaedf182408 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:40.738188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:40.750100 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1273/2569] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:40 2022] PUT /identity/v3/projects/99c41edea3da461dbf21dd5ae8085d22/users/594b1ef30080423a89ff8bbc96da881f/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:00:40.823783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35628bc6-a395-4827-af4c-faaedf182408 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:40.825864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35628bc6-a395-4827-af4c-faaedf182408 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:40.839454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1297/2570] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 08:00:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 26 08:00:40.861678 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d923d-6676-4a65-a793-3c469d970ef0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:40.863577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-942d923d-6676-4a65-a793-3c469d970ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:40.990641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942d923d-6676-4a65-a793-3c469d970ef0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:40.999023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:41.000508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:41.000944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:41.001243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] PATH_INFO: `/v3/users/97b50576f8ef4fe4b67cc631737b8d0b` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:41.023743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b50576f8ef4fe4b67cc631737b8d0b)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:41.028513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:41.143388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b50576f8ef4fe4b67cc631737b8d0b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:41.144303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b50576f8ef4fe4b67cc631737b8d0b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:41.166533 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b50576f8ef4fe4b67cc631737b8d0b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:41.177447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b50576f8ef4fe4b67cc631737b8d0b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:41.200123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b50576f8ef4fe4b67cc631737b8d0b'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:41.210063 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1274/2571] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:40 2022] DELETE /identity/v3/users/97b50576f8ef4fe4b67cc631737b8d0b => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:41.214818 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb2857-f2cc-4c84-92b0-b411b44f7158 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:41.216000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb2857-f2cc-4c84-92b0-b411b44f7158 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:41.216471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb2857-f2cc-4c84-92b0-b411b44f7158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:41.242297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8edb2857-f2cc-4c84-92b0-b411b44f7158 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:41.244082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8edb2857-f2cc-4c84-92b0-b411b44f7158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:41.338387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1298/2572] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:41.348506 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:41.350183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:41.467255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb334b-843b-4123-bee7-41be83a00587 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:41.476966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9379-7b58-4047-81e9-f4c9b0a029b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:41.477731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9379-7b58-4047-81e9-f4c9b0a029b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:41.478294 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9379-7b58-4047-81e9-f4c9b0a029b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:41.479207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9379-7b58-4047-81e9-f4c9b0a029b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:41.498004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e9379-7b58-4047-81e9-f4c9b0a029b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:41.499573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e9379-7b58-4047-81e9-f4c9b0a029b4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:41.517055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1275/2573] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:41.710065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1dffb-b126-401b-9ed6-6b6826ffdbf6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:41.710365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1dffb-b126-401b-9ed6-6b6826ffdbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:41.713858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1dffb-b126-401b-9ed6-6b6826ffdbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:41.769454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1ee1dffb-b126-401b-9ed6-6b6826ffdbf6 None None] MFA Rules not processed for user `594b1ef30080423a89ff8bbc96da881f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:41.770900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1ee1dffb-b126-401b-9ed6-6b6826ffdbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:41.828037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:41.918837 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1299/2574] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:41 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:41.997525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9379-7b58-4047-81e9-f4c9b0a029b4 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:41.999230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-033e9379-7b58-4047-81e9-f4c9b0a029b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:42.029050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9379-7b58-4047-81e9-f4c9b0a029b4 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:42.040179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:42.041124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:42.045869 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:42.046431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] PATH_INFO: `/v3/projects/ebed0772fd2548d3a7c6725944c8d634` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:42.048712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebed0772fd2548d3a7c6725944c8d634)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:42.051394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:42.075476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:42.133481 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebed0772fd2548d3a7c6725944c8d634', 'request_id': 'req-07d09a28-39fa-4e6d-873f-48376ffddcea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:42.150508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebed0772fd2548d3a7c6725944c8d634', 'request_id': 'req-07d09a28-39fa-4e6d-873f-48376ffddcea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:42.165467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebed0772fd2548d3a7c6725944c8d634', 'request_id': 'req-07d09a28-39fa-4e6d-873f-48376ffddcea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:42.189435 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] The token cache is being invalidate because project ebed0772fd2548d3a7c6725944c8d634 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:42.189732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:42.190479 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:42.192306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1276/2575] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:41 2022] DELETE /identity/v3/projects/ebed0772fd2548d3a7c6725944c8d634 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:42.196461 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1dffb-b126-401b-9ed6-6b6826ffdbf6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:42.222457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1dffb-b126-401b-9ed6-6b6826ffdbf6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:42.331812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:42.332926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:42.333725 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:42.334265 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] PATH_INFO: `/v3/users/8914f965db7641679c1c8cc872803c09` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:42.335703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8914f965db7641679c1c8cc872803c09)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:42.337990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:42.404550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8914f965db7641679c1c8cc872803c09'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:42.405273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8914f965db7641679c1c8cc872803c09'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:42.422784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8914f965db7641679c1c8cc872803c09'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:42.451271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8914f965db7641679c1c8cc872803c09'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:42.465106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6506a17f-19e7-4ed9-9bf2-988f6dfed54f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8914f965db7641679c1c8cc872803c09'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:42.485794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1300/2576] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:42 2022] DELETE /identity/v3/users/8914f965db7641679c1c8cc872803c09 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:43.048237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:43.049691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:43.083383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d09a28-39fa-4e6d-873f-48376ffddcea admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:43.189984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:43.203539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:43.204005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:43.204420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] PATH_INFO: `/v3/projects/0b4ab54f3c854671a9b0e8403f902ff0` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:43.206431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b4ab54f3c854671a9b0e8403f902ff0)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:43.208739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:43.284049 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b4ab54f3c854671a9b0e8403f902ff0', 'request_id': 'req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:43.291403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b4ab54f3c854671a9b0e8403f902ff0', 'request_id': 'req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:43.315885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b4ab54f3c854671a9b0e8403f902ff0', 'request_id': 'req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:43.336828 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] The token cache is being invalidate because project 0b4ab54f3c854671a9b0e8403f902ff0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:43.337268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:43.337921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-bc6e46ca-81a1-44d9-b97f-abcbcc08572f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:43.343041 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1277/2577] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:43 2022] DELETE /identity/v3/projects/0b4ab54f3c854671a9b0e8403f902ff0 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:43.345194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06bcc4-0df1-4300-92f4-0806e6f41e3b None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:43.345664 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06bcc4-0df1-4300-92f4-0806e6f41e3b None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:43.345839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c06bcc4-0df1-4300-92f4-0806e6f41e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:43.386310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1c06bcc4-0df1-4300-92f4-0806e6f41e3b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:43.387586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1c06bcc4-0df1-4300-92f4-0806e6f41e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:43.431589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:43.476429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1301/2578] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:43.484885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4729f5-5f8c-4dba-a56c-bdce56175e47 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:43.485271 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4729f5-5f8c-4dba-a56c-bdce56175e47 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:43.485552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4729f5-5f8c-4dba-a56c-bdce56175e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:43.522924 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2a4729f5-5f8c-4dba-a56c-bdce56175e47 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:43.525440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2a4729f5-5f8c-4dba-a56c-bdce56175e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:43.550174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1278/2579] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:43.551841 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06bcc4-0df1-4300-92f4-0806e6f41e3b None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:43.578038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c06bcc4-0df1-4300-92f4-0806e6f41e3b None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:43.583117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f97935-10db-41dd-9e65-112a917eb10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:43.583777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97935-10db-41dd-9e65-112a917eb10d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:43.584054 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97935-10db-41dd-9e65-112a917eb10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:43.584325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f97935-10db-41dd-9e65-112a917eb10d admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:43.585045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f97935-10db-41dd-9e65-112a917eb10d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:43.585348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f97935-10db-41dd-9e65-112a917eb10d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:43.587400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f97935-10db-41dd-9e65-112a917eb10d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:43.617059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1302/2580] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:00:43.617910 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4729f5-5f8c-4dba-a56c-bdce56175e47 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:43.631691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4729f5-5f8c-4dba-a56c-bdce56175e47 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:43.642392 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7fa8e-93c2-4dcb-b5df-cad97b7c70ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:43.643057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7fa8e-93c2-4dcb-b5df-cad97b7c70ed admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:43.643337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7fa8e-93c2-4dcb-b5df-cad97b7c70ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:43.643611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd7fa8e-93c2-4dcb-b5df-cad97b7c70ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:43.644953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7fa8e-93c2-4dcb-b5df-cad97b7c70ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:43.651534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd7fa8e-93c2-4dcb-b5df-cad97b7c70ed admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:43.711685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1279/2581] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:43 2022] POST /identity/v3/projects => generated 379 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:43.715815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f97935-10db-41dd-9e65-112a917eb10d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:43.728508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f97935-10db-41dd-9e65-112a917eb10d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:43.821811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c19d5-904b-4b2f-8e91-8e2d4ba94650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:43.822788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c19d5-904b-4b2f-8e91-8e2d4ba94650 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:43.822788 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c19d5-904b-4b2f-8e91-8e2d4ba94650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:43.823012 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c19d5-904b-4b2f-8e91-8e2d4ba94650 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:43.823706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c19d5-904b-4b2f-8e91-8e2d4ba94650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:43.826293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c19d5-904b-4b2f-8e91-8e2d4ba94650 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:43.868435 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1303/2582] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:43 2022] POST /identity/v3/users => generated 290 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:43.874161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7fa8e-93c2-4dcb-b5df-cad97b7c70ed admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:43.892567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd7fa8e-93c2-4dcb-b5df-cad97b7c70ed admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:43.903954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38677e4d-51c8-4b88-bf7c-85dd61e54477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:43.904705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38677e4d-51c8-4b88-bf7c-85dd61e54477 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:43.904958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38677e4d-51c8-4b88-bf7c-85dd61e54477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:43.905228 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38677e4d-51c8-4b88-bf7c-85dd61e54477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:43.906182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38677e4d-51c8-4b88-bf7c-85dd61e54477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:43.907859 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38677e4d-51c8-4b88-bf7c-85dd61e54477 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:43.913094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:43.931578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1280/2583] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:43.935388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c19d5-904b-4b2f-8e91-8e2d4ba94650 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:43.961049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c19d5-904b-4b2f-8e91-8e2d4ba94650 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:43.984192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38677e4d-51c8-4b88-bf7c-85dd61e54477 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:43.984359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4bdf9-8fb2-4113-9474-e2b193954c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:43.985037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4bdf9-8fb2-4113-9474-e2b193954c56 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:43.985336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4bdf9-8fb2-4113-9474-e2b193954c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:43.985609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c4bdf9-8fb2-4113-9474-e2b193954c56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:43.986335 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4bdf9-8fb2-4113-9474-e2b193954c56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:43.987966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c4bdf9-8fb2-4113-9474-e2b193954c56 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:43.988362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-38677e4d-51c8-4b88-bf7c-85dd61e54477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:44.006914 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38677e4d-51c8-4b88-bf7c-85dd61e54477 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:44.015493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:44.016533 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:44.017150 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:44.017525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] PATH_INFO: `/v3/users/753024d13b614c6ca50942d8b481e79f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:44.019081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=753024d13b614c6ca50942d8b481e79f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:44.020606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1304/2584] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:44.024953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4bdf9-8fb2-4113-9474-e2b193954c56 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:44.025655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:44.046187 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c4bdf9-8fb2-4113-9474-e2b193954c56 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:44.060978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c789023-3d46-4e23-b803-2306b0347c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:44.062191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c789023-3d46-4e23-b803-2306b0347c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:44.062191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c789023-3d46-4e23-b803-2306b0347c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:44.062417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c789023-3d46-4e23-b803-2306b0347c94 admin admin] PATH_INFO: `/v3/projects/3b8de20a399d416792574ee21de1c652/users/5d57be8534504be6b1ad60ecde8978b3/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:44.095456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c789023-3d46-4e23-b803-2306b0347c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b8de20a399d416792574ee21de1c652, user_id=5d57be8534504be6b1ad60ecde8978b3, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:44.098485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c789023-3d46-4e23-b803-2306b0347c94 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:44.110299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '753024d13b614c6ca50942d8b481e79f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.112022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '753024d13b614c6ca50942d8b481e79f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.129131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1305/2585] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:44 2022] PUT /identity/v3/projects/3b8de20a399d416792574ee21de1c652/users/5d57be8534504be6b1ad60ecde8978b3/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:44.140666 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ff5b6-5eca-441f-9061-cb54e2417d82 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:44.140951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ff5b6-5eca-441f-9061-cb54e2417d82 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:44.141219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6ff5b6-5eca-441f-9061-cb54e2417d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:44.147028 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '753024d13b614c6ca50942d8b481e79f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.159891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '753024d13b614c6ca50942d8b481e79f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.174523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '753024d13b614c6ca50942d8b481e79f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.197665 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-dc6ff5b6-5eca-441f-9061-cb54e2417d82 None None] MFA Rules not processed for user `5d57be8534504be6b1ad60ecde8978b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:44.198983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dc6ff5b6-5eca-441f-9061-cb54e2417d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:44.203001 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1281/2586] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:43 2022] DELETE /identity/v3/users/753024d13b614c6ca50942d8b481e79f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:44.364107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1306/2587] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:44.368632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:44.371062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:44.510083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ff5b6-5eca-441f-9061-cb54e2417d82 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:44.511951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-dc6ff5b6-5eca-441f-9061-cb54e2417d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:44.520146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc06093-4aa9-4456-a0fd-ce3db4f046c2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:44.533874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa599e-4d42-4864-ba7f-7ae0f3e1fa58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:44.534708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa599e-4d42-4864-ba7f-7ae0f3e1fa58 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:44.535046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa599e-4d42-4864-ba7f-7ae0f3e1fa58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:44.535390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa599e-4d42-4864-ba7f-7ae0f3e1fa58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:44.536144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa599e-4d42-4864-ba7f-7ae0f3e1fa58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:44.538179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa599e-4d42-4864-ba7f-7ae0f3e1fa58 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:44.549976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6ff5b6-5eca-441f-9061-cb54e2417d82 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:44.561245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:44.561962 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:44.562248 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:44.562525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] PATH_INFO: `/v3/projects/19287120410743e6a7bdc3347c778385` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:44.566698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19287120410743e6a7bdc3347c778385)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:44.567254 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:44.575357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:44.598032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1282/2588] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:44 2022] POST /identity/v3/projects => generated 381 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:00:44.599020 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa599e-4d42-4864-ba7f-7ae0f3e1fa58 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:44.683085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:44.687314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19287120410743e6a7bdc3347c778385', 'request_id': 'req-12a60c05-876a-4dd1-8926-ffc10eebb003'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.699097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa599e-4d42-4864-ba7f-7ae0f3e1fa58 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:44.700474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19287120410743e6a7bdc3347c778385', 'request_id': 'req-12a60c05-876a-4dd1-8926-ffc10eebb003'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.713778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19287120410743e6a7bdc3347c778385', 'request_id': 'req-12a60c05-876a-4dd1-8926-ffc10eebb003'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.741024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] The token cache is being invalidate because project 19287120410743e6a7bdc3347c778385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:44.741321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.744498 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.747369 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1307/2589] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:44 2022] DELETE /identity/v3/projects/19287120410743e6a7bdc3347c778385 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:44.754476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:44.756516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:44.772942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551f4f8-6623-45b5-8f81-46fb346e170c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:44.774344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551f4f8-6623-45b5-8f81-46fb346e170c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:44.774626 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551f4f8-6623-45b5-8f81-46fb346e170c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:44.774903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551f4f8-6623-45b5-8f81-46fb346e170c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:44.778310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551f4f8-6623-45b5-8f81-46fb346e170c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:44.783056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551f4f8-6623-45b5-8f81-46fb346e170c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:44.824953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1283/2590] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:44 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:44.829380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551f4f8-6623-45b5-8f81-46fb346e170c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:44.832727 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b551f4f8-6623-45b5-8f81-46fb346e170c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:44.849283 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a60c05-876a-4dd1-8926-ffc10eebb003 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:44.852182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551f4f8-6623-45b5-8f81-46fb346e170c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:44.858708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:44.859572 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:44.860020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:44.860450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] PATH_INFO: `/v3/users/594b1ef30080423a89ff8bbc96da881f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:44.862001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] RBAC: Authorizing `identity:delete_user(user_id=594b1ef30080423a89ff8bbc96da881f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:44.862686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93586776-9598-4cb0-8c3b-77a90307db6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:44.864244 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93586776-9598-4cb0-8c3b-77a90307db6e admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:44.864575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:44.864715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93586776-9598-4cb0-8c3b-77a90307db6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:44.865201 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93586776-9598-4cb0-8c3b-77a90307db6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:44.866098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93586776-9598-4cb0-8c3b-77a90307db6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:44.869464 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93586776-9598-4cb0-8c3b-77a90307db6e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:44.890133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1284/2591] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:44.896600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93586776-9598-4cb0-8c3b-77a90307db6e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:44.930390 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93586776-9598-4cb0-8c3b-77a90307db6e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:44.941092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255aa3ab-c930-49d7-9f65-45eacb23a35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:44.945683 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255aa3ab-c930-49d7-9f65-45eacb23a35c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:44.946780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255aa3ab-c930-49d7-9f65-45eacb23a35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:44.948498 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255aa3ab-c930-49d7-9f65-45eacb23a35c admin admin] PATH_INFO: `/v3/projects/ab769558e8aa434a8883e6a51f0b9e7e/users/2cceb8d56153434daa9128203ae33978/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:44.961120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '594b1ef30080423a89ff8bbc96da881f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.961870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '594b1ef30080423a89ff8bbc96da881f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:44.976759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255aa3ab-c930-49d7-9f65-45eacb23a35c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab769558e8aa434a8883e6a51f0b9e7e, user_id=2cceb8d56153434daa9128203ae33978, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:44.979698 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255aa3ab-c930-49d7-9f65-45eacb23a35c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:44.990810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '594b1ef30080423a89ff8bbc96da881f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:45.000281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '594b1ef30080423a89ff8bbc96da881f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:45.040938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '594b1ef30080423a89ff8bbc96da881f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:45.049021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1308/2592] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:44 2022] DELETE /identity/v3/users/594b1ef30080423a89ff8bbc96da881f => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:00:45.052886 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1285/2593] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:44 2022] PUT /identity/v3/projects/ab769558e8aa434a8883e6a51f0b9e7e/users/2cceb8d56153434daa9128203ae33978/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:45.066156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:45.148446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:45.237692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:45.325188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcbc041-1e2c-4fba-a692-585cda7cb98b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:45.332798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51d0b4-990e-4583-9a45-53d1ead48808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:45.334227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51d0b4-990e-4583-9a45-53d1ead48808 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:45.334227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51d0b4-990e-4583-9a45-53d1ead48808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:45.334566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51d0b4-990e-4583-9a45-53d1ead48808 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:45.335433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51d0b4-990e-4583-9a45-53d1ead48808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:45.337293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb51d0b4-990e-4583-9a45-53d1ead48808 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:45.355239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1309/2594] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:45.359746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255aa3ab-c930-49d7-9f65-45eacb23a35c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:45.479927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255aa3ab-c930-49d7-9f65-45eacb23a35c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:45.490977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2dae7-f690-4acd-b708-a3fe37fa77cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:45.492018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2dae7-f690-4acd-b708-a3fe37fa77cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:45.492470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2dae7-f690-4acd-b708-a3fe37fa77cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:45.495202 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c2dae7-f690-4acd-b708-a3fe37fa77cb admin admin] PATH_INFO: `/v3/projects/ab769558e8aa434a8883e6a51f0b9e7e/users/2cceb8d56153434daa9128203ae33978/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:45.498530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2dae7-f690-4acd-b708-a3fe37fa77cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab769558e8aa434a8883e6a51f0b9e7e, user_id=2cceb8d56153434daa9128203ae33978, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:45.501368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c2dae7-f690-4acd-b708-a3fe37fa77cb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:45.528964 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1286/2595] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:45 2022] PUT /identity/v3/projects/ab769558e8aa434a8883e6a51f0b9e7e/users/2cceb8d56153434daa9128203ae33978/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:45.531118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccaa23-6cbc-40b1-9670-37a0701a471d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:45.531615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccaa23-6cbc-40b1-9670-37a0701a471d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:45.532655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adccaa23-6cbc-40b1-9670-37a0701a471d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:45.549039 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2dae7-f690-4acd-b708-a3fe37fa77cb admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:45.552216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-65c2dae7-f690-4acd-b708-a3fe37fa77cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:45.631052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-adccaa23-6cbc-40b1-9670-37a0701a471d None None] MFA Rules not processed for user `2cceb8d56153434daa9128203ae33978`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:45.633194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-adccaa23-6cbc-40b1-9670-37a0701a471d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:45.756938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1310/2596] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:45 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:45.765429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccaa23-6cbc-40b1-9670-37a0701a471d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:45.768289 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-adccaa23-6cbc-40b1-9670-37a0701a471d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:45.785527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:45.810253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c2dae7-f690-4acd-b708-a3fe37fa77cb admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:45.819379 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:45.820558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:45.821520 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:45.822003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] PATH_INFO: `/v3/projects/99c41edea3da461dbf21dd5ae8085d22` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:45.823713 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99c41edea3da461dbf21dd5ae8085d22)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:45.826971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:45.888381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adccaa23-6cbc-40b1-9670-37a0701a471d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:45.895580 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12a360-e8e1-452e-adff-19f43da76efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:45.896536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12a360-e8e1-452e-adff-19f43da76efc service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:45.897660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12a360-e8e1-452e-adff-19f43da76efc service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:45.898216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12a360-e8e1-452e-adff-19f43da76efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:45.917367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12a360-e8e1-452e-adff-19f43da76efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:45.919726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12a360-e8e1-452e-adff-19f43da76efc service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:45.924901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99c41edea3da461dbf21dd5ae8085d22', 'request_id': 'req-83ebf33a-8cc1-45b8-8a98-891968344ae5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:45.937247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99c41edea3da461dbf21dd5ae8085d22', 'request_id': 'req-83ebf33a-8cc1-45b8-8a98-891968344ae5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:45.950097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99c41edea3da461dbf21dd5ae8085d22', 'request_id': 'req-83ebf33a-8cc1-45b8-8a98-891968344ae5'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:45.954819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1311/2597] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:00:45.967595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] The token cache is being invalidate because project 99c41edea3da461dbf21dd5ae8085d22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:45.968499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:45.969466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-83ebf33a-8cc1-45b8-8a98-891968344ae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:45.972363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1287/2598] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:45 2022] DELETE /identity/v3/projects/99c41edea3da461dbf21dd5ae8085d22 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:45.977079 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd3243-88da-41b6-a723-fd8de348efca None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:45.977615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd3243-88da-41b6-a723-fd8de348efca None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:45.978745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dd3243-88da-41b6-a723-fd8de348efca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.009130 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c4dd3243-88da-41b6-a723-fd8de348efca None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:46.011587 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c4dd3243-88da-41b6-a723-fd8de348efca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:46.076677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:46.171457 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1312/2599] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:46.182876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66147c1c-8f8d-476b-ac2b-ac92c3091195 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:46.182876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66147c1c-8f8d-476b-ac2b-ac92c3091195 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:46.182876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66147c1c-8f8d-476b-ac2b-ac92c3091195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.225849 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-66147c1c-8f8d-476b-ac2b-ac92c3091195 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:46.227638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-66147c1c-8f8d-476b-ac2b-ac92c3091195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:46.239984 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1288/2600] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:46.244521 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd3243-88da-41b6-a723-fd8de348efca None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:46.261252 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dd3243-88da-41b6-a723-fd8de348efca None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:46.268197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae17f53-cba4-448e-8afd-8dfa3c1f9bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:46.269150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17f53-cba4-448e-8afd-8dfa3c1f9bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:46.269488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17f53-cba4-448e-8afd-8dfa3c1f9bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:46.270949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae17f53-cba4-448e-8afd-8dfa3c1f9bd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.271909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae17f53-cba4-448e-8afd-8dfa3c1f9bd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:46.272268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae17f53-cba4-448e-8afd-8dfa3c1f9bd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:46.275291 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae17f53-cba4-448e-8afd-8dfa3c1f9bd0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:46.298301 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1313/2601] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 08:00:46.305813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66147c1c-8f8d-476b-ac2b-ac92c3091195 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:46.322139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66147c1c-8f8d-476b-ac2b-ac92c3091195 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:46.338281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874f426-4645-4cdc-834f-e196ce58799e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:46.341490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874f426-4645-4cdc-834f-e196ce58799e admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:46.341490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874f426-4645-4cdc-834f-e196ce58799e admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:46.341490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874f426-4645-4cdc-834f-e196ce58799e admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.343166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874f426-4645-4cdc-834f-e196ce58799e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:46.346404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874f426-4645-4cdc-834f-e196ce58799e admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:46.455921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1289/2602] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:46 2022] POST /identity/v3/projects => generated 377 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:46.459374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae17f53-cba4-448e-8afd-8dfa3c1f9bd0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:46.479267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae17f53-cba4-448e-8afd-8dfa3c1f9bd0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:46.566104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874f426-4645-4cdc-834f-e196ce58799e admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:46.568223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f874f426-4645-4cdc-834f-e196ce58799e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:46.603426 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f614693-5c75-40b6-bcfc-e2b781c0e8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:46.605632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f614693-5c75-40b6-bcfc-e2b781c0e8e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:46.605632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f614693-5c75-40b6-bcfc-e2b781c0e8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:46.605632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f614693-5c75-40b6-bcfc-e2b781c0e8e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.605632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f614693-5c75-40b6-bcfc-e2b781c0e8e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:46.607832 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f614693-5c75-40b6-bcfc-e2b781c0e8e9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:46.638770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1314/2603] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:46 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:46.643402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f614693-5c75-40b6-bcfc-e2b781c0e8e9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:46.657573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f614693-5c75-40b6-bcfc-e2b781c0e8e9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:46.665569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f4b74-0e47-423f-94f9-86377ee8cc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:46.666913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f4b74-0e47-423f-94f9-86377ee8cc03 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:46.666913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f4b74-0e47-423f-94f9-86377ee8cc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:46.666913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3f4b74-0e47-423f-94f9-86377ee8cc03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.667597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f4b74-0e47-423f-94f9-86377ee8cc03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:46.669338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3f4b74-0e47-423f-94f9-86377ee8cc03 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:46.677898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874f426-4645-4cdc-834f-e196ce58799e admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:46.685433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735089d-4528-4633-abe1-a703f83958b1 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cceb8d56153434daa9128203ae33978', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab769558e8aa434a8883e6a51f0b9e7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:46.686480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735089d-4528-4633-abe1-a703f83958b1 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:46.686932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735089d-4528-4633-abe1-a703f83958b1 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:46.687090 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1315/2604] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:46.687657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735089d-4528-4633-abe1-a703f83958b1 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.688184 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f4b74-0e47-423f-94f9-86377ee8cc03 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:46.688776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735089d-4528-4633-abe1-a703f83958b1 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:46.689247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735089d-4528-4633-abe1-a703f83958b1 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:46.691646 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1735089d-4528-4633-abe1-a703f83958b1 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:46.700575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3f4b74-0e47-423f-94f9-86377ee8cc03 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:46.707277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebf8d2-7cd0-4895-90f2-a2bb9a30ab52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:46.708077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebf8d2-7cd0-4895-90f2-a2bb9a30ab52 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:46.708554 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebf8d2-7cd0-4895-90f2-a2bb9a30ab52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:46.709500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebf8d2-7cd0-4895-90f2-a2bb9a30ab52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.710392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebf8d2-7cd0-4895-90f2-a2bb9a30ab52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:46.712441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebf8d2-7cd0-4895-90f2-a2bb9a30ab52 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:46.715988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1290/2605] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:00:46.720476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735089d-4528-4633-abe1-a703f83958b1 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:46.724333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1316/2606] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:46.728814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebf8d2-7cd0-4895-90f2-a2bb9a30ab52 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:46.744293 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebf8d2-7cd0-4895-90f2-a2bb9a30ab52 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:46.745242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735089d-4528-4633-abe1-a703f83958b1 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:46.749944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782e3572-3ddb-4184-a17d-b5d6ab64de7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:46.750549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcfb30-c101-4ead-8409-da63f3c2eff5 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cceb8d56153434daa9128203ae33978', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab769558e8aa434a8883e6a51f0b9e7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:46.751187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcfb30-c101-4ead-8409-da63f3c2eff5 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:46.751329 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3572-3ddb-4184-a17d-b5d6ab64de7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:46.751453 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcfb30-c101-4ead-8409-da63f3c2eff5 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:46.751741 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdcfb30-c101-4ead-8409-da63f3c2eff5 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.751875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3572-3ddb-4184-a17d-b5d6ab64de7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:46.752344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782e3572-3ddb-4184-a17d-b5d6ab64de7e admin admin] PATH_INFO: `/v3/projects/0cbc60f9775d425182c62f1003f197fb/users/8fb43f5c4bd744828cb4011a4d8ec210/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.752467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdcfb30-c101-4ead-8409-da63f3c2eff5 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:46.754380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdcfb30-c101-4ead-8409-da63f3c2eff5 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:46.796410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e3572-3ddb-4184-a17d-b5d6ab64de7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cbc60f9775d425182c62f1003f197fb, user_id=8fb43f5c4bd744828cb4011a4d8ec210, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:46.802641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782e3572-3ddb-4184-a17d-b5d6ab64de7e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:46.810275 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1291/2607] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:46 2022] POST /identity/v3/projects => generated 353 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:46.814155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcfb30-c101-4ead-8409-da63f3c2eff5 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:46.856687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1317/2608] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:46 2022] PUT /identity/v3/projects/0cbc60f9775d425182c62f1003f197fb/users/8fb43f5c4bd744828cb4011a4d8ec210/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:46.858614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20464de4-7184-4ef0-a915-039dd6276eb0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:46.858906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20464de4-7184-4ef0-a915-039dd6276eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:46.859232 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20464de4-7184-4ef0-a915-039dd6276eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:46.899555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:46.910724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-20464de4-7184-4ef0-a915-039dd6276eb0 None None] MFA Rules not processed for user `8fb43f5c4bd744828cb4011a4d8ec210`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:46.912029 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-20464de4-7184-4ef0-a915-039dd6276eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:46.969239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdcfb30-c101-4ead-8409-da63f3c2eff5 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:47.005032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:47.013152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1318/2609] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:47.018381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20464de4-7184-4ef0-a915-039dd6276eb0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:47.115460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad27505-5efe-43c6-a5fa-6126b361da30 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cceb8d56153434daa9128203ae33978', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab769558e8aa434a8883e6a51f0b9e7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:47.116428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad27505-5efe-43c6-a5fa-6126b361da30 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:47.116726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad27505-5efe-43c6-a5fa-6126b361da30 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:47.117066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad27505-5efe-43c6-a5fa-6126b361da30 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:47.117832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad27505-5efe-43c6-a5fa-6126b361da30 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:47.118541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad27505-5efe-43c6-a5fa-6126b361da30 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:47.124641 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad27505-5efe-43c6-a5fa-6126b361da30 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:47.146132 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20464de4-7184-4ef0-a915-039dd6276eb0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:47.154777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1292/2610] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 08:00:47.155348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75a2cc-ebcc-4fb5-8c12-c579903eb16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:47.156355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75a2cc-ebcc-4fb5-8c12-c579903eb16c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:47.156642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75a2cc-ebcc-4fb5-8c12-c579903eb16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:47.156924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75a2cc-ebcc-4fb5-8c12-c579903eb16c admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:47.157604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75a2cc-ebcc-4fb5-8c12-c579903eb16c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:47.166255 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a75a2cc-ebcc-4fb5-8c12-c579903eb16c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:47.212813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad27505-5efe-43c6-a5fa-6126b361da30 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:47.230057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad27505-5efe-43c6-a5fa-6126b361da30 tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:47.236933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1319/2611] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:47 2022] POST /identity/v3/projects => generated 377 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:47.240061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75a2cc-ebcc-4fb5-8c12-c579903eb16c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:47.241058 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2cceb8d56153434daa9128203ae33978', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab769558e8aa434a8883e6a51f0b9e7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:47.241752 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:47.242047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:47.242330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] PATH_INFO: `/v3/projects/3603febc6dfb4413a6def860fa275206` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:47.246775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3603febc6dfb4413a6def860fa275206)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:47.249143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:47.332507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3603febc6dfb4413a6def860fa275206', 'request_id': 'req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:47.344465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75a2cc-ebcc-4fb5-8c12-c579903eb16c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:47.345422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3603febc6dfb4413a6def860fa275206', 'request_id': 'req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:47.363513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3603febc6dfb4413a6def860fa275206', 'request_id': 'req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:47.387502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] The token cache is being invalidate because project 3603febc6dfb4413a6def860fa275206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:47.388045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:47.389858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:47.392689 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1293/2612] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:47 2022] DELETE /identity/v3/projects/3603febc6dfb4413a6def860fa275206 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:47.442286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bd9c3-d9e3-4977-a410-0d363bfd78a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:47.443396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bd9c3-d9e3-4977-a410-0d363bfd78a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:47.443671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bd9c3-d9e3-4977-a410-0d363bfd78a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:47.443949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8bd9c3-d9e3-4977-a410-0d363bfd78a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:47.444625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bd9c3-d9e3-4977-a410-0d363bfd78a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:47.447188 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8bd9c3-d9e3-4977-a410-0d363bfd78a7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:47.491075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1320/2613] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:47 2022] POST /identity/v3/users => generated 295 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:47.495538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:47.496808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:47.513148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1e059-c5b7-4e99-8657-5d8e605a67fa tempest-VolumeQuotaClassesTest-1189374113 tempest-VolumeQuotaClassesTest-1189374113-project-admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:47.521692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e28da1-110a-4014-b1d9-df172d3ee64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:47.521692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28da1-110a-4014-b1d9-df172d3ee64d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:47.521692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28da1-110a-4014-b1d9-df172d3ee64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:47.521692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e28da1-110a-4014-b1d9-df172d3ee64d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:47.522450 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e28da1-110a-4014-b1d9-df172d3ee64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:47.524223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e28da1-110a-4014-b1d9-df172d3ee64d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:47.541510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1294/2614] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:47.545965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bd9c3-d9e3-4977-a410-0d363bfd78a7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:47.565118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8bd9c3-d9e3-4977-a410-0d363bfd78a7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:47.571486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f722c59-72cc-458b-a791-2f8175c83d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:47.572328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f722c59-72cc-458b-a791-2f8175c83d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:47.572621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f722c59-72cc-458b-a791-2f8175c83d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:47.572939 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f722c59-72cc-458b-a791-2f8175c83d08 admin admin] PATH_INFO: `/v3/projects/3bf5e93f7a3b4b7189df2d2129fa74a9/users/afb43c6f8ff04f66a2bb27cdfc5e66f0/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:47.625773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f722c59-72cc-458b-a791-2f8175c83d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf5e93f7a3b4b7189df2d2129fa74a9, user_id=afb43c6f8ff04f66a2bb27cdfc5e66f0, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:47.635398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f722c59-72cc-458b-a791-2f8175c83d08 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:47.638892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:47.665405 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1321/2615] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:47 2022] PUT /identity/v3/projects/3bf5e93f7a3b4b7189df2d2129fa74a9/users/afb43c6f8ff04f66a2bb27cdfc5e66f0/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:00:47.669704 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e28da1-110a-4014-b1d9-df172d3ee64d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:47.692368 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e28da1-110a-4014-b1d9-df172d3ee64d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:47.757265 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:47.760835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f722c59-72cc-458b-a791-2f8175c83d08 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:47.763247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f722c59-72cc-458b-a791-2f8175c83d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:47.789330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870c553-51b6-4e5f-ba2e-841a5697b6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:47.790048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870c553-51b6-4e5f-ba2e-841a5697b6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:47.790340 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870c553-51b6-4e5f-ba2e-841a5697b6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:47.790619 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870c553-51b6-4e5f-ba2e-841a5697b6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:47.791330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870c553-51b6-4e5f-ba2e-841a5697b6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:47.793208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870c553-51b6-4e5f-ba2e-841a5697b6f0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:47.804594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1295/2616] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:47.809655 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870c553-51b6-4e5f-ba2e-841a5697b6f0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:47.832153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870c553-51b6-4e5f-ba2e-841a5697b6f0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:47.846063 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b6250-ea0b-41c0-b814-9f8a64530b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:47.847998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b6250-ea0b-41c0-b814-9f8a64530b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:47.848365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b6250-ea0b-41c0-b814-9f8a64530b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:47.849215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b6250-ea0b-41c0-b814-9f8a64530b01 admin admin] PATH_INFO: `/v3/projects/3bf5e93f7a3b4b7189df2d2129fa74a9/users/afb43c6f8ff04f66a2bb27cdfc5e66f0/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:47.851775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b6250-ea0b-41c0-b814-9f8a64530b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bf5e93f7a3b4b7189df2d2129fa74a9, user_id=afb43c6f8ff04f66a2bb27cdfc5e66f0, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:47.857414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b6250-ea0b-41c0-b814-9f8a64530b01 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:47.859082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f722c59-72cc-458b-a791-2f8175c83d08 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:47.866213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:47.868262 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:47.868559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:47.868845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] PATH_INFO: `/v3/users/5d57be8534504be6b1ad60ecde8978b3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:47.870423 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d57be8534504be6b1ad60ecde8978b3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:47.871813 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1296/2617] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:47 2022] PUT /identity/v3/projects/3bf5e93f7a3b4b7189df2d2129fa74a9/users/afb43c6f8ff04f66a2bb27cdfc5e66f0/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:47.884885 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e7cf8-5d6e-48f6-8199-62cb037f629d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:47.885329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e7cf8-5d6e-48f6-8199-62cb037f629d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:47.886217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e7cf8-5d6e-48f6-8199-62cb037f629d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:47.892264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:47.923461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-657e7cf8-5d6e-48f6-8199-62cb037f629d None None] MFA Rules not processed for user `afb43c6f8ff04f66a2bb27cdfc5e66f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:47.924177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-657e7cf8-5d6e-48f6-8199-62cb037f629d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:47.971978 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d57be8534504be6b1ad60ecde8978b3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:47.972565 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d57be8534504be6b1ad60ecde8978b3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:47.991060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d57be8534504be6b1ad60ecde8978b3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:47.999641 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d57be8534504be6b1ad60ecde8978b3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:48.010473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-05e249e3-bf88-4835-b775-404e65fe4370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d57be8534504be6b1ad60ecde8978b3'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:48.021848 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1322/2618] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:47 2022] DELETE /identity/v3/users/5d57be8534504be6b1ad60ecde8978b3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:48.024767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f584e66-3b03-4c40-a8db-17632f5c92c7 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:48.025082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f584e66-3b03-4c40-a8db-17632f5c92c7 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:48.025437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f584e66-3b03-4c40-a8db-17632f5c92c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:48.027558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1297/2619] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:48.041137 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e7cf8-5d6e-48f6-8199-62cb037f629d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:48.042854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-657e7cf8-5d6e-48f6-8199-62cb037f629d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:48.045676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-1f584e66-3b03-4c40-a8db-17632f5c92c7 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:48.046961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f584e66-3b03-4c40-a8db-17632f5c92c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:48.115789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e7cf8-5d6e-48f6-8199-62cb037f629d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:48.116371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1323/2620] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:48.126211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f584e66-3b03-4c40-a8db-17632f5c92c7 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:48.128303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1f584e66-3b03-4c40-a8db-17632f5c92c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:48.147052 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:48.181127 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdf808-3a3c-4d9b-9350-9cfa8a744fb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:48.182253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdf808-3a3c-4d9b-9350-9cfa8a744fb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:48.182253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdf808-3a3c-4d9b-9350-9cfa8a744fb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:48.182611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdf808-3a3c-4d9b-9350-9cfa8a744fb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:48.185750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-56cdf808-3a3c-4d9b-9350-9cfa8a744fb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:48.202756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:48.209564 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f584e66-3b03-4c40-a8db-17632f5c92c7 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:48.214195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8daed-7952-4916-bd6b-7fe2da4e101a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:48.215071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8daed-7952-4916-bd6b-7fe2da4e101a service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:48.215548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8daed-7952-4916-bd6b-7fe2da4e101a service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:48.216004 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8daed-7952-4916-bd6b-7fe2da4e101a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:48.226856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8daed-7952-4916-bd6b-7fe2da4e101a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:48.228758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8daed-7952-4916-bd6b-7fe2da4e101a service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:48.253284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1324/2621] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:48.261574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdf808-3a3c-4d9b-9350-9cfa8a744fb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:48.263474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cdf808-3a3c-4d9b-9350-9cfa8a744fb4 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:48.273483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1298/2622] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:48 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:00:48.629322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8daed-7952-4916-bd6b-7fe2da4e101a service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:48.633110 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d7e8daed-7952-4916-bd6b-7fe2da4e101a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:48.663892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8daed-7952-4916-bd6b-7fe2da4e101a service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:48.668797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:48.669606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:48.670143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:48.670427 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] PATH_INFO: `/v3/projects/3b8de20a399d416792574ee21de1c652` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:48.671611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b8de20a399d416792574ee21de1c652)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:48.673584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:48.748882 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b8de20a399d416792574ee21de1c652', 'request_id': 'req-de69cdaf-daaa-4cea-96ba-06e143d57340'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:48.757393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b8de20a399d416792574ee21de1c652', 'request_id': 'req-de69cdaf-daaa-4cea-96ba-06e143d57340'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:48.766259 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b8de20a399d416792574ee21de1c652', 'request_id': 'req-de69cdaf-daaa-4cea-96ba-06e143d57340'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:48.786867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] The token cache is being invalidate because project 3b8de20a399d416792574ee21de1c652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:48.787164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:48.787921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:48.791205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1325/2623] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:48 2022] DELETE /identity/v3/projects/3b8de20a399d416792574ee21de1c652 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:48.794957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdf808-3a3c-4d9b-9350-9cfa8a744fb4 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:48.810298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cdf808-3a3c-4d9b-9350-9cfa8a744fb4 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:48.910932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:48.912273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:48.912273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:48.912505 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] PATH_INFO: `/v3/users/2cceb8d56153434daa9128203ae33978` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:48.914734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cceb8d56153434daa9128203ae33978)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:48.916573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:48.979374 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cceb8d56153434daa9128203ae33978'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:48.981082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cceb8d56153434daa9128203ae33978'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:48.999671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cceb8d56153434daa9128203ae33978'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:49.008030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cceb8d56153434daa9128203ae33978'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:49.020399 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0301f33f-439d-4c84-b568-35fd72184011 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cceb8d56153434daa9128203ae33978'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:49.031412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1299/2624] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:48 2022] DELETE /identity/v3/users/2cceb8d56153434daa9128203ae33978 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:49.423437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:49.425143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:49.450569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69cdaf-daaa-4cea-96ba-06e143d57340 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:49.529892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:49.529892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:49.529892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:49.529892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] PATH_INFO: `/v3/projects/ab769558e8aa434a8883e6a51f0b9e7e` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:49.530424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab769558e8aa434a8883e6a51f0b9e7e)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:49.532093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:49.583791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:49.596943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab769558e8aa434a8883e6a51f0b9e7e', 'request_id': 'req-ef9e4673-c16e-45e7-8a77-2f2761dabe38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:49.607318 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab769558e8aa434a8883e6a51f0b9e7e', 'request_id': 'req-ef9e4673-c16e-45e7-8a77-2f2761dabe38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:49.614801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab769558e8aa434a8883e6a51f0b9e7e', 'request_id': 'req-ef9e4673-c16e-45e7-8a77-2f2761dabe38'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:49.638951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] The token cache is being invalidate because project ab769558e8aa434a8883e6a51f0b9e7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:49.639987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:49.640618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ef9e4673-c16e-45e7-8a77-2f2761dabe38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:49.642420 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1326/2625] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:49 2022] DELETE /identity/v3/projects/ab769558e8aa434a8883e6a51f0b9e7e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:49.647254 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386529f-b29b-4054-a7c0-de771fcac82b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:49.647569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386529f-b29b-4054-a7c0-de771fcac82b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:49.647854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386529f-b29b-4054-a7c0-de771fcac82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:49.677522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7386529f-b29b-4054-a7c0-de771fcac82b None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:49.679837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7386529f-b29b-4054-a7c0-de771fcac82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:49.722247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:49.757683 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1300/2626] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:49.766127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b0dbe-30bf-41d8-8b96-3b03cb9ed118 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:49.766430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b0dbe-30bf-41d8-8b96-3b03cb9ed118 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:49.766687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b0dbe-30bf-41d8-8b96-3b03cb9ed118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:49.790698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-e67b0dbe-30bf-41d8-8b96-3b03cb9ed118 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:49.791861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e67b0dbe-30bf-41d8-8b96-3b03cb9ed118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:49.815672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386529f-b29b-4054-a7c0-de771fcac82b None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:49.822760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1327/2627] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:49.827282 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386529f-b29b-4054-a7c0-de771fcac82b None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:49.835572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165047ba-df36-4a03-8eef-f07ba72fcfcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:49.836602 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165047ba-df36-4a03-8eef-f07ba72fcfcf admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:49.836892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165047ba-df36-4a03-8eef-f07ba72fcfcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:49.837167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165047ba-df36-4a03-8eef-f07ba72fcfcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:49.838044 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165047ba-df36-4a03-8eef-f07ba72fcfcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:49.838351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165047ba-df36-4a03-8eef-f07ba72fcfcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:49.839947 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165047ba-df36-4a03-8eef-f07ba72fcfcf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:49.858391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1301/2628] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:00:49.859719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b0dbe-30bf-41d8-8b96-3b03cb9ed118 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:49.869290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b0dbe-30bf-41d8-8b96-3b03cb9ed118 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:49.874336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6717f8-affc-4d23-b33f-66b3015faa51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:49.874965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6717f8-affc-4d23-b33f-66b3015faa51 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:49.875239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6717f8-affc-4d23-b33f-66b3015faa51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:49.875509 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6717f8-affc-4d23-b33f-66b3015faa51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:49.876171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6717f8-affc-4d23-b33f-66b3015faa51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:49.878112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6717f8-affc-4d23-b33f-66b3015faa51 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:49.922943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1328/2629] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:49 2022] POST /identity/v3/projects => generated 381 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:49.923292 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165047ba-df36-4a03-8eef-f07ba72fcfcf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:49.937074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165047ba-df36-4a03-8eef-f07ba72fcfcf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:50.016040 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347183d-b1af-45d6-b57f-1cbcf80053a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:50.038788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347183d-b1af-45d6-b57f-1cbcf80053a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.039108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347183d-b1af-45d6-b57f-1cbcf80053a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.045524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347183d-b1af-45d6-b57f-1cbcf80053a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.045524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347183d-b1af-45d6-b57f-1cbcf80053a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:50.055211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347183d-b1af-45d6-b57f-1cbcf80053a6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:50.092874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1302/2630] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:49 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:00:50.094634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6717f8-affc-4d23-b33f-66b3015faa51 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:50.108626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6717f8-affc-4d23-b33f-66b3015faa51 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:50.126228 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f973b8f-1a96-49c1-b294-b2c64f0b84b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:50.126998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f973b8f-1a96-49c1-b294-b2c64f0b84b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.127608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f973b8f-1a96-49c1-b294-b2c64f0b84b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.130394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f973b8f-1a96-49c1-b294-b2c64f0b84b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.131470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f973b8f-1a96-49c1-b294-b2c64f0b84b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:50.133225 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f973b8f-1a96-49c1-b294-b2c64f0b84b2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:50.147916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1329/2631] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:50.149138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347183d-b1af-45d6-b57f-1cbcf80053a6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:50.171070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347183d-b1af-45d6-b57f-1cbcf80053a6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:50.176542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc07f2a-fdda-4154-983f-57881f837790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:50.177544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc07f2a-fdda-4154-983f-57881f837790 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.177953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc07f2a-fdda-4154-983f-57881f837790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.178296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc07f2a-fdda-4154-983f-57881f837790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.179082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc07f2a-fdda-4154-983f-57881f837790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:50.181732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc07f2a-fdda-4154-983f-57881f837790 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:50.197563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1303/2632] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:50.198775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f973b8f-1a96-49c1-b294-b2c64f0b84b2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:50.213348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f973b8f-1a96-49c1-b294-b2c64f0b84b2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:50.218872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81f6c4-0d58-4563-bdea-042b06a92848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:50.219556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81f6c4-0d58-4563-bdea-042b06a92848 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.219838 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81f6c4-0d58-4563-bdea-042b06a92848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.220270 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81f6c4-0d58-4563-bdea-042b06a92848 admin admin] PATH_INFO: `/v3/projects/d2157c0f70834b93a1516c1922ae8fba/users/ff5aa5d603014fcc9120b543275bc970/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.238275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81f6c4-0d58-4563-bdea-042b06a92848 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2157c0f70834b93a1516c1922ae8fba, user_id=ff5aa5d603014fcc9120b543275bc970, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:50.241009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81f6c4-0d58-4563-bdea-042b06a92848 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:50.261808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1330/2633] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:50 2022] PUT /identity/v3/projects/d2157c0f70834b93a1516c1922ae8fba/users/ff5aa5d603014fcc9120b543275bc970/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:50.274154 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bde7e-ca7a-455e-aec4-dda8d03990e5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.274681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bde7e-ca7a-455e-aec4-dda8d03990e5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.275456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0bde7e-ca7a-455e-aec4-dda8d03990e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.309311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2c0bde7e-ca7a-455e-aec4-dda8d03990e5 None None] MFA Rules not processed for user `ff5aa5d603014fcc9120b543275bc970`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:50.310987 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2c0bde7e-ca7a-455e-aec4-dda8d03990e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:50.398819 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1304/2634] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:50.403965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81f6c4-0d58-4563-bdea-042b06a92848 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:50.567822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81f6c4-0d58-4563-bdea-042b06a92848 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:50.585021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82989b5b-7df9-45d5-b2fa-1db16e24c48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:50.586908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82989b5b-7df9-45d5-b2fa-1db16e24c48b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.588214 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82989b5b-7df9-45d5-b2fa-1db16e24c48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.593207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82989b5b-7df9-45d5-b2fa-1db16e24c48b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.594156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82989b5b-7df9-45d5-b2fa-1db16e24c48b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:50.600474 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82989b5b-7df9-45d5-b2fa-1db16e24c48b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:50.635851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1331/2635] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:50 2022] POST /identity/v3/projects => generated 379 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:50.639694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bde7e-ca7a-455e-aec4-dda8d03990e5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:50.656786 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0bde7e-ca7a-455e-aec4-dda8d03990e5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:50.688248 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:50.726812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f6d65-5824-47af-9f81-dd5467e8ff01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:50.727480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f6d65-5824-47af-9f81-dd5467e8ff01 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.727762 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f6d65-5824-47af-9f81-dd5467e8ff01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.728036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f6d65-5824-47af-9f81-dd5467e8ff01 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.728723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f6d65-5824-47af-9f81-dd5467e8ff01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:50.730700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f6d65-5824-47af-9f81-dd5467e8ff01 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:50.757376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1305/2636] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:50 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:50.758988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82989b5b-7df9-45d5-b2fa-1db16e24c48b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:50.771305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82989b5b-7df9-45d5-b2fa-1db16e24c48b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:50.776361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ab776-4e6f-4939-a3df-22c81734adc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:50.776983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ab776-4e6f-4939-a3df-22c81734adc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.777264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ab776-4e6f-4939-a3df-22c81734adc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.777543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782ab776-4e6f-4939-a3df-22c81734adc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.778284 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ab776-4e6f-4939-a3df-22c81734adc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:50.780758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782ab776-4e6f-4939-a3df-22c81734adc9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:50.795693 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1332/2637] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:00:50.797059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f6d65-5824-47af-9f81-dd5467e8ff01 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:50.810906 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f6d65-5824-47af-9f81-dd5467e8ff01 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:50.815972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286e267-c9c2-413d-8c9d-7361f0e916db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:50.817083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286e267-c9c2-413d-8c9d-7361f0e916db admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.817366 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286e267-c9c2-413d-8c9d-7361f0e916db admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.817645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286e267-c9c2-413d-8c9d-7361f0e916db admin admin] PATH_INFO: `/v3/projects/39d37232cd1f41938c42a7b214380142/users/09cc307395754e52aa69d1c2f1593861/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.837184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286e267-c9c2-413d-8c9d-7361f0e916db admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d37232cd1f41938c42a7b214380142, user_id=09cc307395754e52aa69d1c2f1593861, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:50.842617 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286e267-c9c2-413d-8c9d-7361f0e916db admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:50.865566 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1306/2638] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:50 2022] PUT /identity/v3/projects/39d37232cd1f41938c42a7b214380142/users/09cc307395754e52aa69d1c2f1593861/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:50.867093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ab776-4e6f-4939-a3df-22c81734adc9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:50.877894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782ab776-4e6f-4939-a3df-22c81734adc9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:50.927392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:50.952497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a34c634-f5ba-4eed-b8ff-e976004d7d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:50.953165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34c634-f5ba-4eed-b8ff-e976004d7d1d admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.953441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34c634-f5ba-4eed-b8ff-e976004d7d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.953718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a34c634-f5ba-4eed-b8ff-e976004d7d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.954453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34c634-f5ba-4eed-b8ff-e976004d7d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:50.956067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a34c634-f5ba-4eed-b8ff-e976004d7d1d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:50.971975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1333/2639] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:50 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:50.973266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286e267-c9c2-413d-8c9d-7361f0e916db admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:50.984576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286e267-c9c2-413d-8c9d-7361f0e916db admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:50.989323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8b841-87f4-48e5-8930-f927325798b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:50.990028 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8b841-87f4-48e5-8930-f927325798b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:50.990315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8b841-87f4-48e5-8930-f927325798b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:50.990596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8b841-87f4-48e5-8930-f927325798b7 admin admin] PATH_INFO: `/v3/projects/39d37232cd1f41938c42a7b214380142/users/09cc307395754e52aa69d1c2f1593861/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:50.992579 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8b841-87f4-48e5-8930-f927325798b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d37232cd1f41938c42a7b214380142, user_id=09cc307395754e52aa69d1c2f1593861, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:50.995245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8b841-87f4-48e5-8930-f927325798b7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:51.009153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1307/2640] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:50 2022] PUT /identity/v3/projects/39d37232cd1f41938c42a7b214380142/users/09cc307395754e52aa69d1c2f1593861/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:51.021628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8388e2-970e-4182-ad01-22662245a736 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:51.021628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8388e2-970e-4182-ad01-22662245a736 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:51.021628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8388e2-970e-4182-ad01-22662245a736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:51.050179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-cb8388e2-970e-4182-ad01-22662245a736 None None] MFA Rules not processed for user `09cc307395754e52aa69d1c2f1593861`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:51.051774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cb8388e2-970e-4182-ad01-22662245a736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:51.159309 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1334/2641] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:51.171188 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8b841-87f4-48e5-8930-f927325798b7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:51.173147 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4c8b841-87f4-48e5-8930-f927325798b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:51.243053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8b841-87f4-48e5-8930-f927325798b7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:51.249026 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f47c66-0bf7-4c8e-82cf-b68a95b2faaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:51.249770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f47c66-0bf7-4c8e-82cf-b68a95b2faaf service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:51.250116 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f47c66-0bf7-4c8e-82cf-b68a95b2faaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:51.250462 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f47c66-0bf7-4c8e-82cf-b68a95b2faaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:51.260460 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f47c66-0bf7-4c8e-82cf-b68a95b2faaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:51.261976 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f47c66-0bf7-4c8e-82cf-b68a95b2faaf service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:51.274394 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1308/2642] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:51.731792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8388e2-970e-4182-ad01-22662245a736 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:51.809107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8388e2-970e-4182-ad01-22662245a736 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:51.814544 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:51.815501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:51.815787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:51.816065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] PATH_INFO: `/v3/users/ff5aa5d603014fcc9120b543275bc970` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:51.817251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff5aa5d603014fcc9120b543275bc970)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:51.819378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:51.880606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff5aa5d603014fcc9120b543275bc970'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:51.881256 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff5aa5d603014fcc9120b543275bc970'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:51.894420 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:51.896795 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff5aa5d603014fcc9120b543275bc970'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:51.906517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff5aa5d603014fcc9120b543275bc970'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:51.920465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff5aa5d603014fcc9120b543275bc970'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:51.929910 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1335/2643] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:51 2022] DELETE /identity/v3/users/ff5aa5d603014fcc9120b543275bc970 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:51.931852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa88cc-bd59-4019-918e-748f964d05fd None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:51.934334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa88cc-bd59-4019-918e-748f964d05fd None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:51.934735 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa88cc-bd59-4019-918e-748f964d05fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:51.955562 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b2fa88cc-bd59-4019-918e-748f964d05fd None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:51.956825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2fa88cc-bd59-4019-918e-748f964d05fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:52.024313 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1309/2644] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:52.035058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:52.036578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:52.105105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe026fab-b31d-462d-9c8e-19353ac899a9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:52.110057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2235a-f923-479f-bd5e-f68b3ecfec60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:52.110806 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2235a-f923-479f-bd5e-f68b3ecfec60 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:52.111197 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2235a-f923-479f-bd5e-f68b3ecfec60 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:52.111561 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2235a-f923-479f-bd5e-f68b3ecfec60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:52.124656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2235a-f923-479f-bd5e-f68b3ecfec60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:52.127075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2235a-f923-479f-bd5e-f68b3ecfec60 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:52.142025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1336/2645] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:52.453246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa88cc-bd59-4019-918e-748f964d05fd None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:52.454793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b2fa88cc-bd59-4019-918e-748f964d05fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:52.468463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa88cc-bd59-4019-918e-748f964d05fd None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:52.473123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:52.473792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:52.474166 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:52.474362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] PATH_INFO: `/v3/projects/d2157c0f70834b93a1516c1922ae8fba` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:52.475629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2157c0f70834b93a1516c1922ae8fba)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:52.477756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:52.510413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:52.559925 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2157c0f70834b93a1516c1922ae8fba', 'request_id': 'req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:52.565637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2157c0f70834b93a1516c1922ae8fba', 'request_id': 'req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:52.574236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2157c0f70834b93a1516c1922ae8fba', 'request_id': 'req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:52.594039 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] The token cache is being invalidate because project d2157c0f70834b93a1516c1922ae8fba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:52.594397 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:52.595146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:52.597066 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1310/2646] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:52 2022] DELETE /identity/v3/projects/d2157c0f70834b93a1516c1922ae8fba => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:52.601103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2235a-f923-479f-bd5e-f68b3ecfec60 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:52.617774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2235a-f923-479f-bd5e-f68b3ecfec60 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:52.688934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:52.689505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:52.689884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:52.690184 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] PATH_INFO: `/v3/users/09cc307395754e52aa69d1c2f1593861` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:52.691365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=09cc307395754e52aa69d1c2f1593861)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:52.693312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:52.781332 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09cc307395754e52aa69d1c2f1593861'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:52.782030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09cc307395754e52aa69d1c2f1593861'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:52.796002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09cc307395754e52aa69d1c2f1593861'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:52.803082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09cc307395754e52aa69d1c2f1593861'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:52.816372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09cc307395754e52aa69d1c2f1593861'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:52.823272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1337/2647] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:00:52 2022] DELETE /identity/v3/users/09cc307395754e52aa69d1c2f1593861 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:53.104694 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:53.106380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:53.120782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea5c8b4-158a-45d0-89df-e012bcdd7d55 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:53.191855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:53.192391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:53.192524 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.192779 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.193052 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] PATH_INFO: `/v3/projects/39d37232cd1f41938c42a7b214380142` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.193715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:53.194539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39d37232cd1f41938c42a7b214380142)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:53.196550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:53.200273 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:53.246643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39d37232cd1f41938c42a7b214380142', 'request_id': 'req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:53.252639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39d37232cd1f41938c42a7b214380142', 'request_id': 'req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:53.258018 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5134607a-2fd3-4b50-957b-b1979b4909eb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:53.260948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39d37232cd1f41938c42a7b214380142', 'request_id': 'req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:53.272375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] The token cache is being invalidate because project 39d37232cd1f41938c42a7b214380142 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:00:53.272659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:53.273289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-1a974cd8-9b95-4243-9d3a-7f0b24ef4c15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:00:53.275223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1311/2648] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:00:53 2022] DELETE /identity/v3/projects/39d37232cd1f41938c42a7b214380142 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:53.276247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ccbc2-168e-4b12-a36e-0dac66ac286b None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.276533 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ccbc2-168e-4b12-a36e-0dac66ac286b None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.276815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534ccbc2-168e-4b12-a36e-0dac66ac286b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.295057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-534ccbc2-168e-4b12-a36e-0dac66ac286b None None] MFA Rules not processed for user `dc6c885fee5440799beac8cfd3e42822`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:53.296314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-534ccbc2-168e-4b12-a36e-0dac66ac286b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:53.308682 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7bba7-25f4-4871-8ba2-35a73af30d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:53.309311 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7bba7-25f4-4871-8ba2-35a73af30d9b service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.309598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7bba7-25f4-4871-8ba2-35a73af30d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.309911 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7bba7-25f4-4871-8ba2-35a73af30d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.311756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-49f7bba7-25f4-4871-8ba2-35a73af30d9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:53.331424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:53.357709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1312/2649] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:53.359004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c07836-acc0-43a3-9702-466af5ac9896 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.359288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c07836-acc0-43a3-9702-466af5ac9896 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.359556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c07836-acc0-43a3-9702-466af5ac9896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.377449 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7bba7-25f4-4871-8ba2-35a73af30d9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:53.379634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7bba7-25f4-4871-8ba2-35a73af30d9b service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:53.397526 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1338/2650] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:00:53.516504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-81c07836-acc0-43a3-9702-466af5ac9896 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:53.518842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-81c07836-acc0-43a3-9702-466af5ac9896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:53.595176 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1313/2651] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:53.604312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee888dc6-03ec-4390-88e5-7fc13b668b59 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.604798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee888dc6-03ec-4390-88e5-7fc13b668b59 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.605085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee888dc6-03ec-4390-88e5-7fc13b668b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.628623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ee888dc6-03ec-4390-88e5-7fc13b668b59 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:53.629940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ee888dc6-03ec-4390-88e5-7fc13b668b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:53.639439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1339/2652] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:53.644016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c07836-acc0-43a3-9702-466af5ac9896 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:53.654843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c07836-acc0-43a3-9702-466af5ac9896 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:53.659564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10302de8-b24a-4c96-a50d-d1d38aa75b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:53.660237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10302de8-b24a-4c96-a50d-d1d38aa75b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.661106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10302de8-b24a-4c96-a50d-d1d38aa75b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.661106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10302de8-b24a-4c96-a50d-d1d38aa75b24 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.661532 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10302de8-b24a-4c96-a50d-d1d38aa75b24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:00:53.661863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10302de8-b24a-4c96-a50d-d1d38aa75b24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:53.663517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10302de8-b24a-4c96-a50d-d1d38aa75b24 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:53.677818 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1314/2653] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 08:00:53.682909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee888dc6-03ec-4390-88e5-7fc13b668b59 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:53.692985 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee888dc6-03ec-4390-88e5-7fc13b668b59 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:53.699325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92476228-7597-4719-b771-2515bc8d4f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:53.699325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476228-7597-4719-b771-2515bc8d4f57 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.699623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476228-7597-4719-b771-2515bc8d4f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.699947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476228-7597-4719-b771-2515bc8d4f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.701085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92476228-7597-4719-b771-2515bc8d4f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:53.702787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92476228-7597-4719-b771-2515bc8d4f57 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:53.733784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1340/2654] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:53 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:53.737204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10302de8-b24a-4c96-a50d-d1d38aa75b24 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:53.766195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10302de8-b24a-4c96-a50d-d1d38aa75b24 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:53.830303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d84d5-e3e7-4a19-a5ec-0b30499722c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:53.830931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d84d5-e3e7-4a19-a5ec-0b30499722c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.831207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d84d5-e3e7-4a19-a5ec-0b30499722c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.831477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d84d5-e3e7-4a19-a5ec-0b30499722c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.832159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d84d5-e3e7-4a19-a5ec-0b30499722c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:53.834121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d84d5-e3e7-4a19-a5ec-0b30499722c6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:53.863807 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1315/2655] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:53 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:53.864903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92476228-7597-4719-b771-2515bc8d4f57 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:53.875578 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92476228-7597-4719-b771-2515bc8d4f57 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:53.881696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fa6be-1a63-4eb9-a10c-62e52ab188ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:53.882355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fa6be-1a63-4eb9-a10c-62e52ab188ca admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.882631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fa6be-1a63-4eb9-a10c-62e52ab188ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.882904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8fa6be-1a63-4eb9-a10c-62e52ab188ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.883648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fa6be-1a63-4eb9-a10c-62e52ab188ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:53.898357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8fa6be-1a63-4eb9-a10c-62e52ab188ca admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:53.907635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1341/2656] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:53.911578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d84d5-e3e7-4a19-a5ec-0b30499722c6 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:53.922846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d84d5-e3e7-4a19-a5ec-0b30499722c6 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:53.928522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0519ae1f-dabd-4d4e-8f57-6b4a871cd6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:53.929159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519ae1f-dabd-4d4e-8f57-6b4a871cd6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.929433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519ae1f-dabd-4d4e-8f57-6b4a871cd6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.929951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519ae1f-dabd-4d4e-8f57-6b4a871cd6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.930674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519ae1f-dabd-4d4e-8f57-6b4a871cd6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:53.932335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0519ae1f-dabd-4d4e-8f57-6b4a871cd6a2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:53.943799 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1316/2657] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:53.948729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fa6be-1a63-4eb9-a10c-62e52ab188ca admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:53.961023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8fa6be-1a63-4eb9-a10c-62e52ab188ca admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:53.967725 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843e28a-390e-42c4-949b-cad03797c1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:53.969413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843e28a-390e-42c4-949b-cad03797c1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:53.969816 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843e28a-390e-42c4-949b-cad03797c1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:53.970391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843e28a-390e-42c4-949b-cad03797c1d2 admin admin] PATH_INFO: `/v3/projects/f7d2af506b844607a46798eafbd6ccc1/users/fa3a627075af4f8d95464510193edcf9/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:53.988765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843e28a-390e-42c4-949b-cad03797c1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7d2af506b844607a46798eafbd6ccc1, user_id=fa3a627075af4f8d95464510193edcf9, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:53.998093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843e28a-390e-42c4-949b-cad03797c1d2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:54.019782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1342/2658] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:53 2022] PUT /identity/v3/projects/f7d2af506b844607a46798eafbd6ccc1/users/fa3a627075af4f8d95464510193edcf9/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:00:54.024666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3f2f0-1d43-44f0-a27b-7b936e66dc56 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:54.024957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3f2f0-1d43-44f0-a27b-7b936e66dc56 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:54.025230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3f2f0-1d43-44f0-a27b-7b936e66dc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:54.049887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-2fb3f2f0-1d43-44f0-a27b-7b936e66dc56 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:54.052159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-2fb3f2f0-1d43-44f0-a27b-7b936e66dc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:54.108858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:54.134164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1317/2659] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:54.144722 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843e28a-390e-42c4-949b-cad03797c1d2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:54.146417 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a843e28a-390e-42c4-949b-cad03797c1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:54.153919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:54.233047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843e28a-390e-42c4-949b-cad03797c1d2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:54.238264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cade482-38dd-4588-9059-07503dc68218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:54.239027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cade482-38dd-4588-9059-07503dc68218 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:54.239313 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cade482-38dd-4588-9059-07503dc68218 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:54.239660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cade482-38dd-4588-9059-07503dc68218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:54.261155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cade482-38dd-4588-9059-07503dc68218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:54.264095 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cade482-38dd-4588-9059-07503dc68218 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:54.276778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1343/2660] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:54.916710 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db1c26-8b9c-4e43-b676-bc8c2826f3d1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:54.916998 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db1c26-8b9c-4e43-b676-bc8c2826f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:54.917274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23db1c26-8b9c-4e43-b676-bc8c2826f3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:54.946333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-23db1c26-8b9c-4e43-b676-bc8c2826f3d1 None None] MFA Rules not processed for user `fa3a627075af4f8d95464510193edcf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:54.947708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-23db1c26-8b9c-4e43-b676-bc8c2826f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:55.012974 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1318/2661] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:55.017721 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cade482-38dd-4588-9059-07503dc68218 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:55.027871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cade482-38dd-4588-9059-07503dc68218 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:55.032767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0535f-4e83-45b9-8cc6-fc52ef0c6491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:55.033368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0535f-4e83-45b9-8cc6-fc52ef0c6491 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:55.033651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0535f-4e83-45b9-8cc6-fc52ef0c6491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:55.033954 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd0535f-4e83-45b9-8cc6-fc52ef0c6491 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:55.034625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0535f-4e83-45b9-8cc6-fc52ef0c6491 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:55.036536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd0535f-4e83-45b9-8cc6-fc52ef0c6491 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:55.064382 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1344/2662] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:00:55 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:55.071342 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db1c26-8b9c-4e43-b676-bc8c2826f3d1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:55.082701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23db1c26-8b9c-4e43-b676-bc8c2826f3d1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:55.139963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83df10e-b85a-45c1-94fe-6ba836a9463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:55.140712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83df10e-b85a-45c1-94fe-6ba836a9463f admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:55.140840 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83df10e-b85a-45c1-94fe-6ba836a9463f admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:55.141105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83df10e-b85a-45c1-94fe-6ba836a9463f admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:55.141800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83df10e-b85a-45c1-94fe-6ba836a9463f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:55.144123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83df10e-b85a-45c1-94fe-6ba836a9463f admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:55.175270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1319/2663] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:00:55 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:00:55.176718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0535f-4e83-45b9-8cc6-fc52ef0c6491 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:55.187317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd0535f-4e83-45b9-8cc6-fc52ef0c6491 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:55.191951 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13ad7f-d8e9-4c1b-938f-bc3abef5a84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:55.192575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13ad7f-d8e9-4c1b-938f-bc3abef5a84e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:55.192853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13ad7f-d8e9-4c1b-938f-bc3abef5a84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:55.193124 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb13ad7f-d8e9-4c1b-938f-bc3abef5a84e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:55.193863 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13ad7f-d8e9-4c1b-938f-bc3abef5a84e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:55.195468 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb13ad7f-d8e9-4c1b-938f-bc3abef5a84e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:55.207675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1345/2664] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:00:55.213920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83df10e-b85a-45c1-94fe-6ba836a9463f admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:55.227113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83df10e-b85a-45c1-94fe-6ba836a9463f admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:55.231788 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078a2f0-ba67-4a40-88ed-59f37f945d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:55.232463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078a2f0-ba67-4a40-88ed-59f37f945d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:55.232816 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078a2f0-ba67-4a40-88ed-59f37f945d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:55.233083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078a2f0-ba67-4a40-88ed-59f37f945d53 admin admin] PATH_INFO: `/v3/projects/3c1ebbe6b95b42d8b51dd1bc905d45b7/users/77bea908832840c780dae251615ace33/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:55.251352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078a2f0-ba67-4a40-88ed-59f37f945d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1ebbe6b95b42d8b51dd1bc905d45b7, user_id=77bea908832840c780dae251615ace33, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:55.254060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8078a2f0-ba67-4a40-88ed-59f37f945d53 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:55.267729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1320/2665] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:55 2022] PUT /identity/v3/projects/3c1ebbe6b95b42d8b51dd1bc905d45b7/users/77bea908832840c780dae251615ace33/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:00:55.271850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13ad7f-d8e9-4c1b-938f-bc3abef5a84e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:55.281944 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb13ad7f-d8e9-4c1b-938f-bc3abef5a84e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:55.339476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d061a7-2527-41f6-b0d3-f58bca99e5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:55.340115 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d061a7-2527-41f6-b0d3-f58bca99e5da admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:55.340411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d061a7-2527-41f6-b0d3-f58bca99e5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:55.340707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d061a7-2527-41f6-b0d3-f58bca99e5da admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:55.341487 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d061a7-2527-41f6-b0d3-f58bca99e5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:55.343075 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d061a7-2527-41f6-b0d3-f58bca99e5da admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:55.353562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1346/2666] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:00:55.358180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078a2f0-ba67-4a40-88ed-59f37f945d53 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:55.377179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8078a2f0-ba67-4a40-88ed-59f37f945d53 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:55.381669 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9779e79-9a90-4b50-88be-0a2ff23cc0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:55.382363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779e79-9a90-4b50-88be-0a2ff23cc0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:55.382639 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779e79-9a90-4b50-88be-0a2ff23cc0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:55.382914 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9779e79-9a90-4b50-88be-0a2ff23cc0f7 admin admin] PATH_INFO: `/v3/projects/3c1ebbe6b95b42d8b51dd1bc905d45b7/users/77bea908832840c780dae251615ace33/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:55.384827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9779e79-9a90-4b50-88be-0a2ff23cc0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c1ebbe6b95b42d8b51dd1bc905d45b7, user_id=77bea908832840c780dae251615ace33, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:55.387512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9779e79-9a90-4b50-88be-0a2ff23cc0f7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:55.397730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1321/2667] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:00:55 2022] PUT /identity/v3/projects/3c1ebbe6b95b42d8b51dd1bc905d45b7/users/77bea908832840c780dae251615ace33/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:00:56.060787 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d6939-5953-4b6c-b282-8213e33850a6 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:56.061366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d6939-5953-4b6c-b282-8213e33850a6 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:56.061366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d6939-5953-4b6c-b282-8213e33850a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:56.091538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4a2d6939-5953-4b6c-b282-8213e33850a6 None None] MFA Rules not processed for user `77bea908832840c780dae251615ace33`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:56.092925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a2d6939-5953-4b6c-b282-8213e33850a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:56.155676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:56.166204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1347/2668] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:00:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:00:56.179073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9779e79-9a90-4b50-88be-0a2ff23cc0f7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:56.180858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c9779e79-9a90-4b50-88be-0a2ff23cc0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:56.222525 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:00:56.227009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d6939-5953-4b6c-b282-8213e33850a6 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:56.228611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4a2d6939-5953-4b6c-b282-8213e33850a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:56.250548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9779e79-9a90-4b50-88be-0a2ff23cc0f7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:56.254255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8778c19-b89b-473b-b38d-85e33f599d50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:56.254884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8778c19-b89b-473b-b38d-85e33f599d50 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:56.255167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8778c19-b89b-473b-b38d-85e33f599d50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:56.255443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8778c19-b89b-473b-b38d-85e33f599d50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:56.293215 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d6939-5953-4b6c-b282-8213e33850a6 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:56.296872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9749eb-5dcf-44f0-a51d-3c2172f4eee0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:56.297508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9749eb-5dcf-44f0-a51d-3c2172f4eee0 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:56.297820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9749eb-5dcf-44f0-a51d-3c2172f4eee0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:56.298099 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9749eb-5dcf-44f0-a51d-3c2172f4eee0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:56.300093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e9749eb-5dcf-44f0-a51d-3c2172f4eee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:56.320064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8778c19-b89b-473b-b38d-85e33f599d50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:56.321909 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8778c19-b89b-473b-b38d-85e33f599d50 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:56.331916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1322/2669] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:56.383593 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9749eb-5dcf-44f0-a51d-3c2172f4eee0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:56.385207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9749eb-5dcf-44f0-a51d-3c2172f4eee0 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:56.397027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1348/2670] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:57.305761 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cf7a-fbfc-497a-b751-c3cb9d2ecbf8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:57.306329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cf7a-fbfc-497a-b751-c3cb9d2ecbf8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:57.306437 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3cf7a-fbfc-497a-b751-c3cb9d2ecbf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:57.326496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-41f3cf7a-fbfc-497a-b751-c3cb9d2ecbf8 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:00:57.327715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-41f3cf7a-fbfc-497a-b751-c3cb9d2ecbf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:57.383439 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1323/2671] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Jul 26 08:00:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:00:57.395597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9749eb-5dcf-44f0-a51d-3c2172f4eee0 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:57.397061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7e9749eb-5dcf-44f0-a51d-3c2172f4eee0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:57.475770 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9749eb-5dcf-44f0-a51d-3c2172f4eee0 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:57.483599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5e77a-0083-4217-b0c7-6e3a5b75c2d2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:57.484536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5e77a-0083-4217-b0c7-6e3a5b75c2d2 service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:57.484937 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5e77a-0083-4217-b0c7-6e3a5b75c2d2 service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:57.485367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5e77a-0083-4217-b0c7-6e3a5b75c2d2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:57.498341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5e77a-0083-4217-b0c7-6e3a5b75c2d2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:57.500387 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5e77a-0083-4217-b0c7-6e3a5b75c2d2 service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:57.514626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1349/2672] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 08:00:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 08:00:57.533038 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3cf7a-fbfc-497a-b751-c3cb9d2ecbf8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:57.554365 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3cf7a-fbfc-497a-b751-c3cb9d2ecbf8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:57.558906 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb902c-6fe9-4c13-bb5a-23218a8233b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:57.559778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb902c-6fe9-4c13-bb5a-23218a8233b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:57.560238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb902c-6fe9-4c13-bb5a-23218a8233b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:57.560862 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb902c-6fe9-4c13-bb5a-23218a8233b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:57.564373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-57eb902c-6fe9-4c13-bb5a-23218a8233b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:57.581674 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb902c-6fe9-4c13-bb5a-23218a8233b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:57.583521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb902c-6fe9-4c13-bb5a-23218a8233b4 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:57.597586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1324/2673] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:00:57.797510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5e77a-0083-4217-b0c7-6e3a5b75c2d2 service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:00:57.800567 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-30c5e77a-0083-4217-b0c7-6e3a5b75c2d2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:00:57.818835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5e77a-0083-4217-b0c7-6e3a5b75c2d2 service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:00:57.832114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721df986-4dc1-40d7-8bd1-3da8ec15f5eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:00:57.832771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721df986-4dc1-40d7-8bd1-3da8ec15f5eb service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:00:57.833116 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721df986-4dc1-40d7-8bd1-3da8ec15f5eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:00:57.833408 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721df986-4dc1-40d7-8bd1-3da8ec15f5eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:00:57.864675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721df986-4dc1-40d7-8bd1-3da8ec15f5eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:00:57.866182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721df986-4dc1-40d7-8bd1-3da8ec15f5eb service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:00:57.903880 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1350/2674] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:00:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:07.593438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=65 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:07.889502 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:18.097240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb902c-6fe9-4c13-bb5a-23218a8233b4 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:18.098881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-57eb902c-6fe9-4c13-bb5a-23218a8233b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:18.115426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb902c-6fe9-4c13-bb5a-23218a8233b4 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:18.121583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef280c2-a3b4-4149-86b3-844ab808bd28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:18.122250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef280c2-a3b4-4149-86b3-844ab808bd28 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:18.123938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef280c2-a3b4-4149-86b3-844ab808bd28 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:18.123938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef280c2-a3b4-4149-86b3-844ab808bd28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:18.143123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7ef280c2-a3b4-4149-86b3-844ab808bd28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:18.163213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef280c2-a3b4-4149-86b3-844ab808bd28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:18.165094 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef280c2-a3b4-4149-86b3-844ab808bd28 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:18.177790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1325/2675] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:21.999187 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721df986-4dc1-40d7-8bd1-3da8ec15f5eb service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:22.000678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-721df986-4dc1-40d7-8bd1-3da8ec15f5eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:22.069537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721df986-4dc1-40d7-8bd1-3da8ec15f5eb service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:22.074997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:22.075642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:22.075924 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:22.076193 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] PATH_INFO: `/v3/users/8fb43f5c4bd744828cb4011a4d8ec210` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:22.077391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb43f5c4bd744828cb4011a4d8ec210)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:22.079362 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:22.126503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb43f5c4bd744828cb4011a4d8ec210'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.127235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb43f5c4bd744828cb4011a4d8ec210'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.139263 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb43f5c4bd744828cb4011a4d8ec210'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.146120 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb43f5c4bd744828cb4011a4d8ec210'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.154161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb43f5c4bd744828cb4011a4d8ec210'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.159903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1351/2676] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:21 2022] DELETE /identity/v3/users/8fb43f5c4bd744828cb4011a4d8ec210 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:22.164567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e1cd0-30f2-4b9a-be02-99fb50184c1e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:22.165062 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e1cd0-30f2-4b9a-be02-99fb50184c1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:22.165232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e1cd0-30f2-4b9a-be02-99fb50184c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:22.184700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-614e1cd0-30f2-4b9a-be02-99fb50184c1e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:22.185930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-614e1cd0-30f2-4b9a-be02-99fb50184c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:22.244638 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1326/2677] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:22.254552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:22.256359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:22.314465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b5a1ee-77f2-40f4-96a8-1bf09f6335ae admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:22.318464 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc93ed-1ae0-4f06-8820-7e207d28c6ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:22.319081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc93ed-1ae0-4f06-8820-7e207d28c6ff service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:22.319367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc93ed-1ae0-4f06-8820-7e207d28c6ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:22.319628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc93ed-1ae0-4f06-8820-7e207d28c6ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:22.330439 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc93ed-1ae0-4f06-8820-7e207d28c6ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:22.331878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc93ed-1ae0-4f06-8820-7e207d28c6ff service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:22.341611 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1352/2678] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:22.618854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e1cd0-30f2-4b9a-be02-99fb50184c1e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:22.620605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-614e1cd0-30f2-4b9a-be02-99fb50184c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:22.633022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e1cd0-30f2-4b9a-be02-99fb50184c1e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:22.638081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:22.638731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:22.639005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:22.639270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] PATH_INFO: `/v3/projects/0cbc60f9775d425182c62f1003f197fb` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:22.641079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cbc60f9775d425182c62f1003f197fb)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:22.643427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:22.708801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cbc60f9775d425182c62f1003f197fb', 'request_id': 'req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.711343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cbc60f9775d425182c62f1003f197fb', 'request_id': 'req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.721157 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cbc60f9775d425182c62f1003f197fb', 'request_id': 'req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.733695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] The token cache is being invalidate because project 0cbc60f9775d425182c62f1003f197fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:22.734239 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.734837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.739607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1327/2679] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:22 2022] DELETE /identity/v3/projects/0cbc60f9775d425182c62f1003f197fb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:22.740675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc93ed-1ae0-4f06-8820-7e207d28c6ff service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:22.751835 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc93ed-1ae0-4f06-8820-7e207d28c6ff service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:22.788428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:22.841233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:22.841947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:22.842280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:22.842540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] PATH_INFO: `/v3/users/afb43c6f8ff04f66a2bb27cdfc5e66f0` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:22.843780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] RBAC: Authorizing `identity:delete_user(user_id=afb43c6f8ff04f66a2bb27cdfc5e66f0)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:22.845913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:22.913233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afb43c6f8ff04f66a2bb27cdfc5e66f0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.914198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afb43c6f8ff04f66a2bb27cdfc5e66f0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.928667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afb43c6f8ff04f66a2bb27cdfc5e66f0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.936642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afb43c6f8ff04f66a2bb27cdfc5e66f0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.948866 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d753e14-5ac6-4f6e-a42d-86cda6b16aed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afb43c6f8ff04f66a2bb27cdfc5e66f0'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:22.961009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1353/2680] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:22 2022] DELETE /identity/v3/users/afb43c6f8ff04f66a2bb27cdfc5e66f0 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:23.304067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:23.306480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:23.335929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c577a-2e47-40aa-91ad-e7f5d8d2b620 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:23.409825 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:23.410556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:23.410908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:23.411255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] PATH_INFO: `/v3/projects/3bf5e93f7a3b4b7189df2d2129fa74a9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:23.412420 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bf5e93f7a3b4b7189df2d2129fa74a9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:23.414573 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:23.426164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:23.467945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bf5e93f7a3b4b7189df2d2129fa74a9', 'request_id': 'req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:23.473543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bf5e93f7a3b4b7189df2d2129fa74a9', 'request_id': 'req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:23.481814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bf5e93f7a3b4b7189df2d2129fa74a9', 'request_id': 'req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:23.495689 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] The token cache is being invalidate because project 3bf5e93f7a3b4b7189df2d2129fa74a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:23.495948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:23.496580 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52aec19e-cff8-4acd-9d57-48abcdb8e6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:23.498143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1328/2681] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:23 2022] DELETE /identity/v3/projects/3bf5e93f7a3b4b7189df2d2129fa74a9 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:23.503482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711702f-ac0b-4154-a4c8-718072d20d1f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:23.503982 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711702f-ac0b-4154-a4c8-718072d20d1f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:23.504102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8711702f-ac0b-4154-a4c8-718072d20d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:23.524479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8711702f-ac0b-4154-a4c8-718072d20d1f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:23.525631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8711702f-ac0b-4154-a4c8-718072d20d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:23.588169 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1354/2682] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:23.596237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b5701-00d0-40c4-aeb7-2e1dca715d52 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:23.596529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b5701-00d0-40c4-aeb7-2e1dca715d52 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:23.596808 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b5701-00d0-40c4-aeb7-2e1dca715d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:23.617422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-9c2b5701-00d0-40c4-aeb7-2e1dca715d52 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:23.618668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9c2b5701-00d0-40c4-aeb7-2e1dca715d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:23.627158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1329/2683] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:23.631560 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711702f-ac0b-4154-a4c8-718072d20d1f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:23.641493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8711702f-ac0b-4154-a4c8-718072d20d1f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:23.647179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306128f8-7d13-4293-bfab-2ce1264f7dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:23.647795 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306128f8-7d13-4293-bfab-2ce1264f7dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:23.648106 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306128f8-7d13-4293-bfab-2ce1264f7dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:23.648378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306128f8-7d13-4293-bfab-2ce1264f7dcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:23.649083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306128f8-7d13-4293-bfab-2ce1264f7dcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:01:23.649350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306128f8-7d13-4293-bfab-2ce1264f7dcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:23.651024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306128f8-7d13-4293-bfab-2ce1264f7dcd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:23.663896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1355/2684] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:01:23.668270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b5701-00d0-40c4-aeb7-2e1dca715d52 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:23.677663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b5701-00d0-40c4-aeb7-2e1dca715d52 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:23.682875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c739443-dd6b-4bd0-82f4-5c2ee7bb5a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:23.683472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c739443-dd6b-4bd0-82f4-5c2ee7bb5a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:23.683763 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c739443-dd6b-4bd0-82f4-5c2ee7bb5a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:23.684003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c739443-dd6b-4bd0-82f4-5c2ee7bb5a5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:23.684700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c739443-dd6b-4bd0-82f4-5c2ee7bb5a5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:23.686660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c739443-dd6b-4bd0-82f4-5c2ee7bb5a5b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:23.721065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1330/2685] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:23 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:23.725508 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306128f8-7d13-4293-bfab-2ce1264f7dcd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:23.735799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306128f8-7d13-4293-bfab-2ce1264f7dcd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:23.761434 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:23.794391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8eb99-f61b-446e-b675-6e49007b5289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:23.794933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8eb99-f61b-446e-b675-6e49007b5289 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:23.795217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8eb99-f61b-446e-b675-6e49007b5289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:23.795494 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8eb99-f61b-446e-b675-6e49007b5289 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:23.796156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8eb99-f61b-446e-b675-6e49007b5289 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:23.798133 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a8eb99-f61b-446e-b675-6e49007b5289 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:23.820009 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1356/2686] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:23 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:23.824744 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c739443-dd6b-4bd0-82f4-5c2ee7bb5a5b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:23.835146 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c739443-dd6b-4bd0-82f4-5c2ee7bb5a5b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:23.840867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef8ac9-4fcb-4dd6-99b1-37f363428077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:23.841473 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8ac9-4fcb-4dd6-99b1-37f363428077 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:23.841712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8ac9-4fcb-4dd6-99b1-37f363428077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:23.842048 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef8ac9-4fcb-4dd6-99b1-37f363428077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:23.842739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef8ac9-4fcb-4dd6-99b1-37f363428077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:23.844329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef8ac9-4fcb-4dd6-99b1-37f363428077 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:23.855243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1331/2687] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:23.859441 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8eb99-f61b-446e-b675-6e49007b5289 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:23.869781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8eb99-f61b-446e-b675-6e49007b5289 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:23.875103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa24a7c-ca0f-44e0-963c-c5deb511a892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:23.875712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa24a7c-ca0f-44e0-963c-c5deb511a892 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:23.875997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa24a7c-ca0f-44e0-963c-c5deb511a892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:23.876276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa24a7c-ca0f-44e0-963c-c5deb511a892 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:23.877182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa24a7c-ca0f-44e0-963c-c5deb511a892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:23.878819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa24a7c-ca0f-44e0-963c-c5deb511a892 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:23.888956 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1357/2688] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:23.893300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef8ac9-4fcb-4dd6-99b1-37f363428077 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:23.903798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef8ac9-4fcb-4dd6-99b1-37f363428077 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:23.909122 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92ad27-499f-44aa-8884-39f7dabc8372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:23.909776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92ad27-499f-44aa-8884-39f7dabc8372 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:23.910085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92ad27-499f-44aa-8884-39f7dabc8372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:23.910344 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92ad27-499f-44aa-8884-39f7dabc8372 admin admin] PATH_INFO: `/v3/projects/f4e0d81eadb44948a3be540b96eb5db5/users/363eface9ae747608dafe6477dfa51c7/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:23.925564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92ad27-499f-44aa-8884-39f7dabc8372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e0d81eadb44948a3be540b96eb5db5, user_id=363eface9ae747608dafe6477dfa51c7, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:23.928253 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92ad27-499f-44aa-8884-39f7dabc8372 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:23.948624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1332/2689] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:23 2022] PUT /identity/v3/projects/f4e0d81eadb44948a3be540b96eb5db5/users/363eface9ae747608dafe6477dfa51c7/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:23.965796 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b36a7-611f-4877-9cb0-3fe7ccff123d None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:23.966112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b36a7-611f-4877-9cb0-3fe7ccff123d None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:23.966375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b36a7-611f-4877-9cb0-3fe7ccff123d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:23.996191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3d6b36a7-611f-4877-9cb0-3fe7ccff123d None None] MFA Rules not processed for user `363eface9ae747608dafe6477dfa51c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:23.997424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3d6b36a7-611f-4877-9cb0-3fe7ccff123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:24.062219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1358/2690] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:24.066851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92ad27-499f-44aa-8884-39f7dabc8372 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.131518 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92ad27-499f-44aa-8884-39f7dabc8372 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.136549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cde06-6b07-4e9b-b949-29b906fd4c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.137161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cde06-6b07-4e9b-b949-29b906fd4c6d admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.137439 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cde06-6b07-4e9b-b949-29b906fd4c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.137717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3cde06-6b07-4e9b-b949-29b906fd4c6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.138396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cde06-6b07-4e9b-b949-29b906fd4c6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.140268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3cde06-6b07-4e9b-b949-29b906fd4c6d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.169195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1333/2691] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:24 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:24.173489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b36a7-611f-4877-9cb0-3fe7ccff123d None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.184516 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b36a7-611f-4877-9cb0-3fe7ccff123d None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.242350 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e9850-c30a-439e-9068-cfee3d593ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.242975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e9850-c30a-439e-9068-cfee3d593ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.243253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e9850-c30a-439e-9068-cfee3d593ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.243534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e9850-c30a-439e-9068-cfee3d593ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.244186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e9850-c30a-439e-9068-cfee3d593ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.246148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e9850-c30a-439e-9068-cfee3d593ee6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.269138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1359/2692] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:24 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:24.273334 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cde06-6b07-4e9b-b949-29b906fd4c6d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.283747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3cde06-6b07-4e9b-b949-29b906fd4c6d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.290133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad099f01-a460-4cca-9b15-1741598cbc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.290782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad099f01-a460-4cca-9b15-1741598cbc51 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.291086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad099f01-a460-4cca-9b15-1741598cbc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.291377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad099f01-a460-4cca-9b15-1741598cbc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.292080 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad099f01-a460-4cca-9b15-1741598cbc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.293661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad099f01-a460-4cca-9b15-1741598cbc51 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.304551 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1334/2693] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:24.308712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e9850-c30a-439e-9068-cfee3d593ee6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.319310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e9850-c30a-439e-9068-cfee3d593ee6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.325150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6afc01-fe97-4f36-80f1-922e72ba87a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.325763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6afc01-fe97-4f36-80f1-922e72ba87a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.326084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6afc01-fe97-4f36-80f1-922e72ba87a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.326358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6afc01-fe97-4f36-80f1-922e72ba87a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.327059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6afc01-fe97-4f36-80f1-922e72ba87a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.328663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6afc01-fe97-4f36-80f1-922e72ba87a7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.339060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1360/2694] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:24.343283 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad099f01-a460-4cca-9b15-1741598cbc51 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.353895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad099f01-a460-4cca-9b15-1741598cbc51 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.360246 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac5dba-5e75-42bd-87d3-afd2baf982a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.360996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac5dba-5e75-42bd-87d3-afd2baf982a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.361274 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac5dba-5e75-42bd-87d3-afd2baf982a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.361542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac5dba-5e75-42bd-87d3-afd2baf982a2 admin admin] PATH_INFO: `/v3/projects/0fd8da4b4be145ad88ab6aca910493a3/users/b36d842174954f4f83c8cb34b07f1cd9/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.370151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:24.379092 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac5dba-5e75-42bd-87d3-afd2baf982a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fd8da4b4be145ad88ab6aca910493a3, user_id=b36d842174954f4f83c8cb34b07f1cd9, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.382194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ac5dba-5e75-42bd-87d3-afd2baf982a2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.397167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1335/2695] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:24 2022] PUT /identity/v3/projects/0fd8da4b4be145ad88ab6aca910493a3/users/b36d842174954f4f83c8cb34b07f1cd9/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:24.407539 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c10b9c-36b3-414e-8c69-19c0dde41c76 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.408101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c10b9c-36b3-414e-8c69-19c0dde41c76 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.408376 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c10b9c-36b3-414e-8c69-19c0dde41c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.459655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f1c10b9c-36b3-414e-8c69-19c0dde41c76 None None] MFA Rules not processed for user `b36d842174954f4f83c8cb34b07f1cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:24.461006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f1c10b9c-36b3-414e-8c69-19c0dde41c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:24.465392 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:24.528341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1361/2696] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:24.533214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac5dba-5e75-42bd-87d3-afd2baf982a2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.597430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac5dba-5e75-42bd-87d3-afd2baf982a2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.602373 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d454602-22f8-479b-9eb7-a8ee5f83be16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.602980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d454602-22f8-479b-9eb7-a8ee5f83be16 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.603258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d454602-22f8-479b-9eb7-a8ee5f83be16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.603529 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d454602-22f8-479b-9eb7-a8ee5f83be16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.604195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d454602-22f8-479b-9eb7-a8ee5f83be16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.606122 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d454602-22f8-479b-9eb7-a8ee5f83be16 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.639163 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1336/2697] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:24 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:24.643399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c10b9c-36b3-414e-8c69-19c0dde41c76 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.654268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c10b9c-36b3-414e-8c69-19c0dde41c76 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.718747 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c1b63-e691-4f0f-9777-24172f6baad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.719617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c1b63-e691-4f0f-9777-24172f6baad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.719987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c1b63-e691-4f0f-9777-24172f6baad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.720458 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c1b63-e691-4f0f-9777-24172f6baad1 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.721167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c1b63-e691-4f0f-9777-24172f6baad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.723242 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339c1b63-e691-4f0f-9777-24172f6baad1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.747632 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1362/2698] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:24 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:24.752317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d454602-22f8-479b-9eb7-a8ee5f83be16 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.763353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d454602-22f8-479b-9eb7-a8ee5f83be16 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.769372 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f92828-6df0-4f1e-b904-27d044c29653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.770081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f92828-6df0-4f1e-b904-27d044c29653 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.770350 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f92828-6df0-4f1e-b904-27d044c29653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.770622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f92828-6df0-4f1e-b904-27d044c29653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.771317 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f92828-6df0-4f1e-b904-27d044c29653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.772916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f92828-6df0-4f1e-b904-27d044c29653 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.784069 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1337/2699] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:24.788537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c1b63-e691-4f0f-9777-24172f6baad1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.798606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c1b63-e691-4f0f-9777-24172f6baad1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.803315 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259f16-1a39-45ef-b4f4-5322c96dc029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.803976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259f16-1a39-45ef-b4f4-5322c96dc029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.804257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259f16-1a39-45ef-b4f4-5322c96dc029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.804538 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259f16-1a39-45ef-b4f4-5322c96dc029 admin admin] PATH_INFO: `/v3/projects/c91559e3406944fa910f3dadd1cab92a/users/12c47960b52345c981aee69705ed11bf/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.819136 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6259f16-1a39-45ef-b4f4-5322c96dc029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91559e3406944fa910f3dadd1cab92a, user_id=12c47960b52345c981aee69705ed11bf, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.822224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6259f16-1a39-45ef-b4f4-5322c96dc029 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.836814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1363/2700] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:24 2022] PUT /identity/v3/projects/c91559e3406944fa910f3dadd1cab92a/users/12c47960b52345c981aee69705ed11bf/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:24.840693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f92828-6df0-4f1e-b904-27d044c29653 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.850596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f92828-6df0-4f1e-b904-27d044c29653 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.908013 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e104cd-c28b-4a0b-81f9-1c0981182c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.908630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e104cd-c28b-4a0b-81f9-1c0981182c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.908908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e104cd-c28b-4a0b-81f9-1c0981182c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.909178 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e104cd-c28b-4a0b-81f9-1c0981182c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.909943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e104cd-c28b-4a0b-81f9-1c0981182c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.911550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e104cd-c28b-4a0b-81f9-1c0981182c09 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.921587 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1338/2701] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:24.926008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259f16-1a39-45ef-b4f4-5322c96dc029 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:24.936974 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259f16-1a39-45ef-b4f4-5322c96dc029 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:24.942200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1623f-4e4d-404e-9515-f65954083400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:24.942858 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1623f-4e4d-404e-9515-f65954083400 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.943140 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1623f-4e4d-404e-9515-f65954083400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.943410 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1623f-4e4d-404e-9515-f65954083400 admin admin] PATH_INFO: `/v3/projects/c91559e3406944fa910f3dadd1cab92a/users/12c47960b52345c981aee69705ed11bf/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:24.945960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1623f-4e4d-404e-9515-f65954083400 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c91559e3406944fa910f3dadd1cab92a, user_id=12c47960b52345c981aee69705ed11bf, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:24.948591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf1623f-4e4d-404e-9515-f65954083400 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:24.959782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1364/2702] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:24 2022] PUT /identity/v3/projects/c91559e3406944fa910f3dadd1cab92a/users/12c47960b52345c981aee69705ed11bf/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:24.969970 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e76a27-d8fe-43e3-a8bc-4503a196dae5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:24.970258 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e76a27-d8fe-43e3-a8bc-4503a196dae5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:24.970540 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e76a27-d8fe-43e3-a8bc-4503a196dae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:25.002705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-31e76a27-d8fe-43e3-a8bc-4503a196dae5 None None] MFA Rules not processed for user `12c47960b52345c981aee69705ed11bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:25.004574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-31e76a27-d8fe-43e3-a8bc-4503a196dae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:25.069381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:25.097834 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1339/2703] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:25.109887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1623f-4e4d-404e-9515-f65954083400 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:25.111943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-cdf1623f-4e4d-404e-9515-f65954083400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:25.189210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf1623f-4e4d-404e-9515-f65954083400 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:25.193781 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cf0a2-4c77-46e7-abba-6dd9700906d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:25.194418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cf0a2-4c77-46e7-abba-6dd9700906d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:25.194701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cf0a2-4c77-46e7-abba-6dd9700906d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:25.194968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cf0a2-4c77-46e7-abba-6dd9700906d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:25.205503 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cf0a2-4c77-46e7-abba-6dd9700906d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:25.206996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cf0a2-4c77-46e7-abba-6dd9700906d4 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:25.212746 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:25.218877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1365/2704] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:25.782688 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e76a27-d8fe-43e3-a8bc-4503a196dae5 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:25.784382 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-31e76a27-d8fe-43e3-a8bc-4503a196dae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:25.797661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e76a27-d8fe-43e3-a8bc-4503a196dae5 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:25.802330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb8e5f-78fe-43e3-9e98-ebb5be499c9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:25.803064 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb8e5f-78fe-43e3-9e98-ebb5be499c9f service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:25.803343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb8e5f-78fe-43e3-9e98-ebb5be499c9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:25.803608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb8e5f-78fe-43e3-9e98-ebb5be499c9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:25.861982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb8e5f-78fe-43e3-9e98-ebb5be499c9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:25.863404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb8e5f-78fe-43e3-9e98-ebb5be499c9f service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:25.872422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1340/2705] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:28.203491 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cf0a2-4c77-46e7-abba-6dd9700906d4 service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:28.204999 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ba5cf0a2-4c77-46e7-abba-6dd9700906d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:28.290862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cf0a2-4c77-46e7-abba-6dd9700906d4 service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:28.297095 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1390e-4c72-4a91-9a83-b8247eccf19e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:28.297626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1390e-4c72-4a91-9a83-b8247eccf19e service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:28.297943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1390e-4c72-4a91-9a83-b8247eccf19e service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:28.298246 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b1390e-4c72-4a91-9a83-b8247eccf19e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:28.300142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-23b1390e-4c72-4a91-9a83-b8247eccf19e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:28.312465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1390e-4c72-4a91-9a83-b8247eccf19e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:28.314200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b1390e-4c72-4a91-9a83-b8247eccf19e service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:28.325488 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1366/2706] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:28.856169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb8e5f-78fe-43e3-9e98-ebb5be499c9f service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:28.857977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7afb8e5f-78fe-43e3-9e98-ebb5be499c9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:28.929547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb8e5f-78fe-43e3-9e98-ebb5be499c9f service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:28.934599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:28.935424 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:28.935768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:28.936106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] PATH_INFO: `/v3/users/fa3a627075af4f8d95464510193edcf9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:28.937290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa3a627075af4f8d95464510193edcf9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:28.939324 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:28.989282 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa3a627075af4f8d95464510193edcf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:28.989958 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa3a627075af4f8d95464510193edcf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.004389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa3a627075af4f8d95464510193edcf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.012011 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa3a627075af4f8d95464510193edcf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.021169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa3a627075af4f8d95464510193edcf9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.023676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:29.029719 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1341/2707] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:28 2022] DELETE /identity/v3/users/fa3a627075af4f8d95464510193edcf9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:29.219515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1390e-4c72-4a91-9a83-b8247eccf19e service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:29.230398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b1390e-4c72-4a91-9a83-b8247eccf19e service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:29.290241 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:29.291013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:29.291288 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:29.291569 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] PATH_INFO: `/v3/projects/f7d2af506b844607a46798eafbd6ccc1` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:29.292876 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7d2af506b844607a46798eafbd6ccc1)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:29.295024 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:29.343989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7d2af506b844607a46798eafbd6ccc1', 'request_id': 'req-0a7ece40-98f2-4b52-af12-dba2ee63aab4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.350077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7d2af506b844607a46798eafbd6ccc1', 'request_id': 'req-0a7ece40-98f2-4b52-af12-dba2ee63aab4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.355864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7d2af506b844607a46798eafbd6ccc1', 'request_id': 'req-0a7ece40-98f2-4b52-af12-dba2ee63aab4'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.373705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] The token cache is being invalidate because project f7d2af506b844607a46798eafbd6ccc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:29.374020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.374663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.376219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1367/2708] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:29 2022] DELETE /identity/v3/projects/f7d2af506b844607a46798eafbd6ccc1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:29.380705 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:29.382134 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:29.395568 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b88feb-3602-497d-8ca0-e0ee25cbc797 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:29.463012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:29.463664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:29.463942 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:29.464216 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] PATH_INFO: `/v3/users/77bea908832840c780dae251615ace33` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:29.465417 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] RBAC: Authorizing `identity:delete_user(user_id=77bea908832840c780dae251615ace33)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:29.467369 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:29.514643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77bea908832840c780dae251615ace33'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.515284 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77bea908832840c780dae251615ace33'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.526935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77bea908832840c780dae251615ace33'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.534190 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77bea908832840c780dae251615ace33'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.543876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-db18f38d-1ee3-499b-9c34-74a95da3efae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77bea908832840c780dae251615ace33'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.550406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1342/2709] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:29 2022] DELETE /identity/v3/users/77bea908832840c780dae251615ace33 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:29.730440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:29.731897 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:29.751853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ece40-98f2-4b52-af12-dba2ee63aab4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:29.772764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:29.821085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:29.821757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:29.822045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:29.822312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] PATH_INFO: `/v3/projects/3c1ebbe6b95b42d8b51dd1bc905d45b7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:29.823542 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c1ebbe6b95b42d8b51dd1bc905d45b7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:29.825482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:29.883780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c1ebbe6b95b42d8b51dd1bc905d45b7', 'request_id': 'req-f7e2471b-0279-4309-b72e-fc340c692d80'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.883780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c1ebbe6b95b42d8b51dd1bc905d45b7', 'request_id': 'req-f7e2471b-0279-4309-b72e-fc340c692d80'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.885609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c1ebbe6b95b42d8b51dd1bc905d45b7', 'request_id': 'req-f7e2471b-0279-4309-b72e-fc340c692d80'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.900123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] The token cache is being invalidate because project 3c1ebbe6b95b42d8b51dd1bc905d45b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:29.900407 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.901129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f7e2471b-0279-4309-b72e-fc340c692d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:29.902757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1368/2710] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:29 2022] DELETE /identity/v3/projects/3c1ebbe6b95b42d8b51dd1bc905d45b7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:29.907851 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af66cd-33e7-42c4-9c50-d237a98bc45a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:29.908403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af66cd-33e7-42c4-9c50-d237a98bc45a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:29.908514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0af66cd-33e7-42c4-9c50-d237a98bc45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:29.928078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-a0af66cd-33e7-42c4-9c50-d237a98bc45a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:29.929267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a0af66cd-33e7-42c4-9c50-d237a98bc45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:29.992645 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1343/2711] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:29.996823 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edbfc6-e587-4b61-9172-5320126d0c12 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:29.997112 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edbfc6-e587-4b61-9172-5320126d0c12 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:29.997411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edbfc6-e587-4b61-9172-5320126d0c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.000903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af66cd-33e7-42c4-9c50-d237a98bc45a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.002618 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-a0af66cd-33e7-42c4-9c50-d237a98bc45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:30.017988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-55edbfc6-e587-4b61-9172-5320126d0c12 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:30.019176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-55edbfc6-e587-4b61-9172-5320126d0c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:30.023411 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:30.027205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1369/2712] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:30.031476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edbfc6-e587-4b61-9172-5320126d0c12 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.040453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edbfc6-e587-4b61-9172-5320126d0c12 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.045731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77586ceb-24d1-4b15-afc4-9c5610283830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.046373 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77586ceb-24d1-4b15-afc4-9c5610283830 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.046654 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77586ceb-24d1-4b15-afc4-9c5610283830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.046925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77586ceb-24d1-4b15-afc4-9c5610283830 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.047626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77586ceb-24d1-4b15-afc4-9c5610283830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:01:30.047929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77586ceb-24d1-4b15-afc4-9c5610283830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.049555 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77586ceb-24d1-4b15-afc4-9c5610283830 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.061547 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1370/2713] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 08:01:30.061716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0af66cd-33e7-42c4-9c50-d237a98bc45a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.065676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20009923-239e-40b8-8c7d-5be667c1108c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.066483 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20009923-239e-40b8-8c7d-5be667c1108c service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.066775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20009923-239e-40b8-8c7d-5be667c1108c service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.067053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20009923-239e-40b8-8c7d-5be667c1108c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.067443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77586ceb-24d1-4b15-afc4-9c5610283830 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.069119 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-20009923-239e-40b8-8c7d-5be667c1108c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:30.077768 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77586ceb-24d1-4b15-afc4-9c5610283830 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.082691 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8288d49-0d3c-460e-b9fa-684206601b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.083275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8288d49-0d3c-460e-b9fa-684206601b11 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.084003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8288d49-0d3c-460e-b9fa-684206601b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.084003 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8288d49-0d3c-460e-b9fa-684206601b11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.084496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8288d49-0d3c-460e-b9fa-684206601b11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.086440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8288d49-0d3c-460e-b9fa-684206601b11 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.116831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1371/2714] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:30 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:30.121167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8288d49-0d3c-460e-b9fa-684206601b11 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.151967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20009923-239e-40b8-8c7d-5be667c1108c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.153393 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20009923-239e-40b8-8c7d-5be667c1108c service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.166979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1344/2715] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:01:30.196760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8288d49-0d3c-460e-b9fa-684206601b11 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.202397 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76046ff2-f24a-4848-a2dc-13d6b4a1ad40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.203053 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76046ff2-f24a-4848-a2dc-13d6b4a1ad40 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.203330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76046ff2-f24a-4848-a2dc-13d6b4a1ad40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.203598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76046ff2-f24a-4848-a2dc-13d6b4a1ad40 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.204258 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76046ff2-f24a-4848-a2dc-13d6b4a1ad40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.206272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76046ff2-f24a-4848-a2dc-13d6b4a1ad40 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.223715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:30.233183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1372/2716] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:30 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:30.237236 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20009923-239e-40b8-8c7d-5be667c1108c service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.248756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20009923-239e-40b8-8c7d-5be667c1108c service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.256378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef738ad0-c2da-4ca6-bce3-7d505a631331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.257292 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef738ad0-c2da-4ca6-bce3-7d505a631331 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.257663 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef738ad0-c2da-4ca6-bce3-7d505a631331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.258087 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef738ad0-c2da-4ca6-bce3-7d505a631331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.262422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef738ad0-c2da-4ca6-bce3-7d505a631331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.264377 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef738ad0-c2da-4ca6-bce3-7d505a631331 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.276502 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1345/2717] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:30.281077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76046ff2-f24a-4848-a2dc-13d6b4a1ad40 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.297704 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76046ff2-f24a-4848-a2dc-13d6b4a1ad40 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.303701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b78de-5ea7-4b83-a8b2-45b70c2bdcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.304334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b78de-5ea7-4b83-a8b2-45b70c2bdcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.304669 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b78de-5ea7-4b83-a8b2-45b70c2bdcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.304945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b78de-5ea7-4b83-a8b2-45b70c2bdcc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.305651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b78de-5ea7-4b83-a8b2-45b70c2bdcc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.307321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b78de-5ea7-4b83-a8b2-45b70c2bdcc3 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.322302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1373/2718] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:30.323486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef738ad0-c2da-4ca6-bce3-7d505a631331 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.337413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef738ad0-c2da-4ca6-bce3-7d505a631331 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.342832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ddb35-f4e5-4e03-be4d-4d36043f2f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.343620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ddb35-f4e5-4e03-be4d-4d36043f2f7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.343903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ddb35-f4e5-4e03-be4d-4d36043f2f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.344177 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59ddb35-f4e5-4e03-be4d-4d36043f2f7c admin admin] PATH_INFO: `/v3/projects/7c22fc1cc86841b3bd226ab15f8b6568/users/cd41363c5b3c4dfaba3ee905fa7242bb/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.362235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ddb35-f4e5-4e03-be4d-4d36043f2f7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c22fc1cc86841b3bd226ab15f8b6568, user_id=cd41363c5b3c4dfaba3ee905fa7242bb, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.366475 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59ddb35-f4e5-4e03-be4d-4d36043f2f7c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.389879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1346/2719] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:30 2022] PUT /identity/v3/projects/7c22fc1cc86841b3bd226ab15f8b6568/users/cd41363c5b3c4dfaba3ee905fa7242bb/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:30.404712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e152c-790b-4ef2-b3ab-38b1a77396fb None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.405095 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e152c-790b-4ef2-b3ab-38b1a77396fb None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.405450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e152c-790b-4ef2-b3ab-38b1a77396fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.449020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-5f7e152c-790b-4ef2-b3ab-38b1a77396fb None None] MFA Rules not processed for user `cd41363c5b3c4dfaba3ee905fa7242bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:30.450300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5f7e152c-790b-4ef2-b3ab-38b1a77396fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:30.513977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ddb35-f4e5-4e03-be4d-4d36043f2f7c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.517268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d59ddb35-f4e5-4e03-be4d-4d36043f2f7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:30.538948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1374/2720] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:30.543676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e152c-790b-4ef2-b3ab-38b1a77396fb None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.586697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59ddb35-f4e5-4e03-be4d-4d36043f2f7c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.593208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.594036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.594198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.594427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] PATH_INFO: `/v3/users/d0e0204de4304b3aaa7c867ea49628ea` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.595857 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e0204de4304b3aaa7c867ea49628ea)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.597837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.623125 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e152c-790b-4ef2-b3ab-38b1a77396fb None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.629045 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4244c-7f50-4237-9ed7-75d1a3b59ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.629763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4244c-7f50-4237-9ed7-75d1a3b59ac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.630123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4244c-7f50-4237-9ed7-75d1a3b59ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.630455 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e4244c-7f50-4237-9ed7-75d1a3b59ac0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.631184 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4244c-7f50-4237-9ed7-75d1a3b59ac0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.633191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e4244c-7f50-4237-9ed7-75d1a3b59ac0 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.648319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:30.654874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e0204de4304b3aaa7c867ea49628ea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:30.655899 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e0204de4304b3aaa7c867ea49628ea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:30.666789 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1375/2721] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:30 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:30.670952 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4244c-7f50-4237-9ed7-75d1a3b59ac0 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.681577 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e0204de4304b3aaa7c867ea49628ea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:30.697654 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e0204de4304b3aaa7c867ea49628ea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:30.710419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-20804452-17ec-4f96-a232-3abaad36a0c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e0204de4304b3aaa7c867ea49628ea'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:30.718977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1347/2722] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:30 2022] DELETE /identity/v3/users/d0e0204de4304b3aaa7c867ea49628ea => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:30.723355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631e01b-1419-40fd-8d0d-32fac5bcc673 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.723709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631e01b-1419-40fd-8d0d-32fac5bcc673 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.724212 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631e01b-1419-40fd-8d0d-32fac5bcc673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.742069 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e4244c-7f50-4237-9ed7-75d1a3b59ac0 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.748738 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:30.749191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f631e01b-1419-40fd-8d0d-32fac5bcc673 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:30.750468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f631e01b-1419-40fd-8d0d-32fac5bcc673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:30.823277 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccfad9-bf88-4ac9-9dd1-306e4fe4c4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.824002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccfad9-bf88-4ac9-9dd1-306e4fe4c4e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.824330 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccfad9-bf88-4ac9-9dd1-306e4fe4c4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.824583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ccfad9-bf88-4ac9-9dd1-306e4fe4c4e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.825601 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccfad9-bf88-4ac9-9dd1-306e4fe4c4e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.827918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ccfad9-bf88-4ac9-9dd1-306e4fe4c4e1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.838996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1348/2723] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:30.851321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631e01b-1419-40fd-8d0d-32fac5bcc673 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.853273 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f631e01b-1419-40fd-8d0d-32fac5bcc673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:30.862589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1376/2724] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:30 2022] POST /identity/v3/users => generated 294 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:30.866965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccfad9-bf88-4ac9-9dd1-306e4fe4c4e1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.868790 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c9ccfad9-bf88-4ac9-9dd1-306e4fe4c4e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:30.882300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ccfad9-bf88-4ac9-9dd1-306e4fe4c4e1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.887101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb1c97-3a3c-449f-abea-3bcb22a653cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.887714 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb1c97-3a3c-449f-abea-3bcb22a653cd admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.887997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb1c97-3a3c-449f-abea-3bcb22a653cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.888264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb1c97-3a3c-449f-abea-3bcb22a653cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.888966 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb1c97-3a3c-449f-abea-3bcb22a653cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.890596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecb1c97-3a3c-449f-abea-3bcb22a653cd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.900964 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1377/2725] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:30.905223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb1c97-3a3c-449f-abea-3bcb22a653cd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.915484 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecb1c97-3a3c-449f-abea-3bcb22a653cd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.920756 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb5c5-2a80-4a79-83be-a86b59e661b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.921890 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb5c5-2a80-4a79-83be-a86b59e661b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.922401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb5c5-2a80-4a79-83be-a86b59e661b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.922701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfcb5c5-2a80-4a79-83be-a86b59e661b2 admin admin] PATH_INFO: `/v3/projects/db087410b4e249b883ef3352de322c17/users/841aa3257ceb4d379e00ca51e881d379/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.923919 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631e01b-1419-40fd-8d0d-32fac5bcc673 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:30.928703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cd45b-98c6-4fd7-944d-e299b6aecab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:30.929314 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cd45b-98c6-4fd7-944d-e299b6aecab9 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:30.929596 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cd45b-98c6-4fd7-944d-e299b6aecab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:30.929906 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8cd45b-98c6-4fd7-944d-e299b6aecab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:30.939768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cd45b-98c6-4fd7-944d-e299b6aecab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.940236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcb5c5-2a80-4a79-83be-a86b59e661b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db087410b4e249b883ef3352de322c17, user_id=841aa3257ceb4d379e00ca51e881d379, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:30.941187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8cd45b-98c6-4fd7-944d-e299b6aecab9 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.944013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfcb5c5-2a80-4a79-83be-a86b59e661b2 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:30.951467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1349/2726] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:01:30.967940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1378/2727] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:30 2022] PUT /identity/v3/projects/db087410b4e249b883ef3352de322c17/users/841aa3257ceb4d379e00ca51e881d379/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:30.981890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cd45b-98c6-4fd7-944d-e299b6aecab9 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:30.994854 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8cd45b-98c6-4fd7-944d-e299b6aecab9 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:31.061089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28e2b8-4937-4ec1-aa8e-b8c103cc3d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:31.061750 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28e2b8-4937-4ec1-aa8e-b8c103cc3d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:31.062045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28e2b8-4937-4ec1-aa8e-b8c103cc3d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:31.062322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28e2b8-4937-4ec1-aa8e-b8c103cc3d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:31.063089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28e2b8-4937-4ec1-aa8e-b8c103cc3d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:31.064889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28e2b8-4937-4ec1-aa8e-b8c103cc3d85 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:31.067738 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:31.076730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1350/2728] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:31.083644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb5c5-2a80-4a79-83be-a86b59e661b2 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:31.100658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfcb5c5-2a80-4a79-83be-a86b59e661b2 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:31.108942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6592c4-ea3f-4606-a215-98dc4e420d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:31.109617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6592c4-ea3f-4606-a215-98dc4e420d9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:31.110414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6592c4-ea3f-4606-a215-98dc4e420d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:31.110414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6592c4-ea3f-4606-a215-98dc4e420d9a admin admin] PATH_INFO: `/v3/projects/db087410b4e249b883ef3352de322c17/users/841aa3257ceb4d379e00ca51e881d379/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:31.112927 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6592c4-ea3f-4606-a215-98dc4e420d9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=db087410b4e249b883ef3352de322c17, user_id=841aa3257ceb4d379e00ca51e881d379, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:31.115799 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6592c4-ea3f-4606-a215-98dc4e420d9a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:31.137833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1379/2729] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:31 2022] PUT /identity/v3/projects/db087410b4e249b883ef3352de322c17/users/841aa3257ceb4d379e00ca51e881d379/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:31.149513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caae29-0d34-4574-9fb1-247e19ad5b50 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:31.150508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caae29-0d34-4574-9fb1-247e19ad5b50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:31.151504 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caae29-0d34-4574-9fb1-247e19ad5b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:31.191289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f1caae29-0d34-4574-9fb1-247e19ad5b50 None None] MFA Rules not processed for user `841aa3257ceb4d379e00ca51e881d379`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:31.192517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f1caae29-0d34-4574-9fb1-247e19ad5b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:31.266843 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1351/2730] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:31.278419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6592c4-ea3f-4606-a215-98dc4e420d9a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:31.280014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caae29-0d34-4574-9fb1-247e19ad5b50 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:31.280549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3c6592c4-ea3f-4606-a215-98dc4e420d9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:31.281574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f1caae29-0d34-4574-9fb1-247e19ad5b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:31.340762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6592c4-ea3f-4606-a215-98dc4e420d9a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:31.344419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ee24f-fdfb-46a0-9db6-2f709c90884c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:31.345044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ee24f-fdfb-46a0-9db6-2f709c90884c service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:31.345404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ee24f-fdfb-46a0-9db6-2f709c90884c service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:31.345676 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ee24f-fdfb-46a0-9db6-2f709c90884c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:31.347482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8d6ee24f-fdfb-46a0-9db6-2f709c90884c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:31.352293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caae29-0d34-4574-9fb1-247e19ad5b50 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:31.356903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:31.357551 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:31.357884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:31.358164 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] PATH_INFO: `/v3/projects/d775913087b54deb9aefe7cd5179a01a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:31.359339 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d775913087b54deb9aefe7cd5179a01a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:31.361286 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:31.409677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd775913087b54deb9aefe7cd5179a01a', 'request_id': 'req-5deee493-b37a-47f0-8cd4-8c55fedfb55d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:31.410400 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ee24f-fdfb-46a0-9db6-2f709c90884c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:31.411903 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ee24f-fdfb-46a0-9db6-2f709c90884c service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:31.415222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd775913087b54deb9aefe7cd5179a01a', 'request_id': 'req-5deee493-b37a-47f0-8cd4-8c55fedfb55d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:31.421173 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1380/2731] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:31.423887 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd775913087b54deb9aefe7cd5179a01a', 'request_id': 'req-5deee493-b37a-47f0-8cd4-8c55fedfb55d'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:31.436778 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] The token cache is being invalidate because project d775913087b54deb9aefe7cd5179a01a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:31.437070 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:31.438106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:31.440055 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1352/2732] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:31 2022] DELETE /identity/v3/projects/d775913087b54deb9aefe7cd5179a01a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:31.444071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ee24f-fdfb-46a0-9db6-2f709c90884c service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:31.451347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:31.456047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ee24f-fdfb-46a0-9db6-2f709c90884c service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:31.531049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:31.531607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:31.531888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:31.532154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] PATH_INFO: `/v3/users/a36dfedd99064b08abbaf3ddc4b10ba2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:31.533436 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a36dfedd99064b08abbaf3ddc4b10ba2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:31.535471 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:31.586420 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a36dfedd99064b08abbaf3ddc4b10ba2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:31.587205 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a36dfedd99064b08abbaf3ddc4b10ba2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:31.601761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a36dfedd99064b08abbaf3ddc4b10ba2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:31.609102 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a36dfedd99064b08abbaf3ddc4b10ba2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:31.617493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0cc6b589-d11c-4a49-8a47-85936335f216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a36dfedd99064b08abbaf3ddc4b10ba2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:31.623804 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1381/2733] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:31 2022] DELETE /identity/v3/users/a36dfedd99064b08abbaf3ddc4b10ba2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:31.908293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:31.909955 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:31.923491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deee493-b37a-47f0-8cd4-8c55fedfb55d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:31.932877 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:31.982430 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:31.983075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:31.983355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:31.983632 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] PATH_INFO: `/v3/projects/764f8182f42341f29ba66b31f5c7b29c` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:31.984773 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=764f8182f42341f29ba66b31f5c7b29c)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:31.986757 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.041009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '764f8182f42341f29ba66b31f5c7b29c', 'request_id': 'req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:32.046714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '764f8182f42341f29ba66b31f5c7b29c', 'request_id': 'req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:32.055174 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '764f8182f42341f29ba66b31f5c7b29c', 'request_id': 'req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:32.068389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] The token cache is being invalidate because project 764f8182f42341f29ba66b31f5c7b29c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:32.068682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:32.069337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-a4cc0a6b-c793-44a4-a680-9c4f6e44be44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:32.071801 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1353/2734] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:31 2022] DELETE /identity/v3/projects/764f8182f42341f29ba66b31f5c7b29c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:32.076913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a03bf1-dc6c-4718-aafd-fe5a705cf48f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.077032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a03bf1-dc6c-4718-aafd-fe5a705cf48f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.077282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a03bf1-dc6c-4718-aafd-fe5a705cf48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.099165 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d9a03bf1-dc6c-4718-aafd-fe5a705cf48f None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:32.100493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d9a03bf1-dc6c-4718-aafd-fe5a705cf48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:32.160233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1382/2735] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:32.168673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7a365-7ec2-49f4-837a-c0956e5dac74 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.168968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7a365-7ec2-49f4-837a-c0956e5dac74 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.169247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7a365-7ec2-49f4-837a-c0956e5dac74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.190085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b5f7a365-7ec2-49f4-837a-c0956e5dac74 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:32.191299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b5f7a365-7ec2-49f4-837a-c0956e5dac74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:32.199703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1354/2736] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:32.204046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a03bf1-dc6c-4718-aafd-fe5a705cf48f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.214853 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a03bf1-dc6c-4718-aafd-fe5a705cf48f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.220297 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fae5b4-157e-4d84-846e-9c35b67201f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.221108 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fae5b4-157e-4d84-846e-9c35b67201f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.221391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fae5b4-157e-4d84-846e-9c35b67201f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.221655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fae5b4-157e-4d84-846e-9c35b67201f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.222384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fae5b4-157e-4d84-846e-9c35b67201f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:01:32.222685 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fae5b4-157e-4d84-846e-9c35b67201f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.224280 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fae5b4-157e-4d84-846e-9c35b67201f6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.241223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1383/2737] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:01:32.242404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7a365-7ec2-49f4-837a-c0956e5dac74 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.251806 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7a365-7ec2-49f4-837a-c0956e5dac74 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.256461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7b3b4-308c-4744-a8de-69902f11b88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.257075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7b3b4-308c-4744-a8de-69902f11b88b admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.257351 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7b3b4-308c-4744-a8de-69902f11b88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.257624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f7b3b4-308c-4744-a8de-69902f11b88b admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.258333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7b3b4-308c-4744-a8de-69902f11b88b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.260197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f7b3b4-308c-4744-a8de-69902f11b88b admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.289558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1355/2738] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:32 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:32.294072 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fae5b4-157e-4d84-846e-9c35b67201f6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.305032 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fae5b4-157e-4d84-846e-9c35b67201f6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.314357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:32.367794 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a2b14-732e-4977-91d3-64d41d129e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.368456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a2b14-732e-4977-91d3-64d41d129e49 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.368728 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a2b14-732e-4977-91d3-64d41d129e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.368998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a2b14-732e-4977-91d3-64d41d129e49 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.369655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a2b14-732e-4977-91d3-64d41d129e49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.371609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a2b14-732e-4977-91d3-64d41d129e49 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.400886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1384/2739] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:32 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:32.402205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7b3b4-308c-4744-a8de-69902f11b88b admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.412114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a2b14-732e-4977-91d3-64d41d129e49 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.413904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f7b3b4-308c-4744-a8de-69902f11b88b admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.414073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-299a2b14-732e-4977-91d3-64d41d129e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:32.418610 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecd5a0-e4af-4be6-86c3-37ef5d328d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.419237 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecd5a0-e4af-4be6-86c3-37ef5d328d77 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.419517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecd5a0-e4af-4be6-86c3-37ef5d328d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.419791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecd5a0-e4af-4be6-86c3-37ef5d328d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.420490 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ecd5a0-e4af-4be6-86c3-37ef5d328d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.422120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ecd5a0-e4af-4be6-86c3-37ef5d328d77 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.426659 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a2b14-732e-4977-91d3-64d41d129e49 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.435452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1356/2740] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:32.435631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.435631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.435631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.435631 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] PATH_INFO: `/v3/users/363eface9ae747608dafe6477dfa51c7` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.436013 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=363eface9ae747608dafe6477dfa51c7)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.436476 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecd5a0-e4af-4be6-86c3-37ef5d328d77 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.437454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.446734 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ecd5a0-e4af-4be6-86c3-37ef5d328d77 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.451290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93bc41-8180-40e1-8120-e5de3c2d2e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.451911 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93bc41-8180-40e1-8120-e5de3c2d2e32 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.452191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93bc41-8180-40e1-8120-e5de3c2d2e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.452466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf93bc41-8180-40e1-8120-e5de3c2d2e32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.453195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93bc41-8180-40e1-8120-e5de3c2d2e32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.454880 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf93bc41-8180-40e1-8120-e5de3c2d2e32 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.465154 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1357/2741] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:32.469892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93bc41-8180-40e1-8120-e5de3c2d2e32 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.479605 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf93bc41-8180-40e1-8120-e5de3c2d2e32 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.484181 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938916c-28a0-4ae1-ac3a-f30092ff52d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.484827 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938916c-28a0-4ae1-ac3a-f30092ff52d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.485097 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938916c-28a0-4ae1-ac3a-f30092ff52d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.485388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938916c-28a0-4ae1-ac3a-f30092ff52d6 admin admin] PATH_INFO: `/v3/projects/6f323246401f468f9ac0f926c64d36c6/users/fac7aa9bd3a549acae77f72a0ca4e2db/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.487396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '363eface9ae747608dafe6477dfa51c7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:32.488314 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '363eface9ae747608dafe6477dfa51c7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:32.500779 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '363eface9ae747608dafe6477dfa51c7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:32.501700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938916c-28a0-4ae1-ac3a-f30092ff52d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f323246401f468f9ac0f926c64d36c6, user_id=fac7aa9bd3a549acae77f72a0ca4e2db, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.504532 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938916c-28a0-4ae1-ac3a-f30092ff52d6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.508171 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '363eface9ae747608dafe6477dfa51c7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:32.516014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-91a78d30-9282-4931-b5af-55a9c8bfc8fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '363eface9ae747608dafe6477dfa51c7'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:32.518599 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1358/2742] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:32 2022] PUT /identity/v3/projects/6f323246401f468f9ac0f926c64d36c6/users/fac7aa9bd3a549acae77f72a0ca4e2db/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:32.522066 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1385/2743] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:32 2022] DELETE /identity/v3/users/363eface9ae747608dafe6477dfa51c7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:32.526422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793b590-20ed-4cb3-aa24-8fd8090f6b50 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.526724 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793b590-20ed-4cb3-aa24-8fd8090f6b50 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.526995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793b590-20ed-4cb3-aa24-8fd8090f6b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.529002 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca10f7-c0fa-4906-97a1-4b8e3a7ccfb2 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.529287 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca10f7-c0fa-4906-97a1-4b8e3a7ccfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.529562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dca10f7-c0fa-4906-97a1-4b8e3a7ccfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.548754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f793b590-20ed-4cb3-aa24-8fd8090f6b50 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:32.549928 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f793b590-20ed-4cb3-aa24-8fd8090f6b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:32.558153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-2dca10f7-c0fa-4906-97a1-4b8e3a7ccfb2 None None] MFA Rules not processed for user `fac7aa9bd3a549acae77f72a0ca4e2db`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:32.559316 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2dca10f7-c0fa-4906-97a1-4b8e3a7ccfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:32.576059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:32.613037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1359/2744] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:32.623077 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793b590-20ed-4cb3-aa24-8fd8090f6b50 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.624446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f793b590-20ed-4cb3-aa24-8fd8090f6b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:32.626295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1386/2745] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:32.630758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca10f7-c0fa-4906-97a1-4b8e3a7ccfb2 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.632460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2dca10f7-c0fa-4906-97a1-4b8e3a7ccfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:32.641152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:32.647556 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dca10f7-c0fa-4906-97a1-4b8e3a7ccfb2 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.653557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006dd40-9b2d-4fa0-90d1-ef7bb2075eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.654212 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006dd40-9b2d-4fa0-90d1-ef7bb2075eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.654492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006dd40-9b2d-4fa0-90d1-ef7bb2075eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.654767 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006dd40-9b2d-4fa0-90d1-ef7bb2075eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.655429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006dd40-9b2d-4fa0-90d1-ef7bb2075eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.657344 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006dd40-9b2d-4fa0-90d1-ef7bb2075eb9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.680995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793b590-20ed-4cb3-aa24-8fd8090f6b50 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.684591 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2689645-8289-47ed-8243-5b6294d6b3dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.685207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2689645-8289-47ed-8243-5b6294d6b3dc service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.685356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1387/2746] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:32 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:32.685469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2689645-8289-47ed-8243-5b6294d6b3dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.685766 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2689645-8289-47ed-8243-5b6294d6b3dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.689312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006dd40-9b2d-4fa0-90d1-ef7bb2075eb9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.696108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2689645-8289-47ed-8243-5b6294d6b3dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.697585 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2689645-8289-47ed-8243-5b6294d6b3dc service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.707245 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1360/2747] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:01:32.758814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006dd40-9b2d-4fa0-90d1-ef7bb2075eb9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.764711 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccbbff-691d-435c-a44d-7050d972164a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.765630 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccbbff-691d-435c-a44d-7050d972164a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.765991 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccbbff-691d-435c-a44d-7050d972164a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.766081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ccbbff-691d-435c-a44d-7050d972164a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.766748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccbbff-691d-435c-a44d-7050d972164a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.768703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ccbbff-691d-435c-a44d-7050d972164a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.798298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1388/2748] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:32 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:32.802946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2689645-8289-47ed-8243-5b6294d6b3dc service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.814491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2689645-8289-47ed-8243-5b6294d6b3dc service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.821624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec59b6-5c2a-4db1-9f4e-cdf443c6d42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.822717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec59b6-5c2a-4db1-9f4e-cdf443c6d42c admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.823004 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec59b6-5c2a-4db1-9f4e-cdf443c6d42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.823287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec59b6-5c2a-4db1-9f4e-cdf443c6d42c admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.824161 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec59b6-5c2a-4db1-9f4e-cdf443c6d42c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.825907 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec59b6-5c2a-4db1-9f4e-cdf443c6d42c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.841078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1361/2749] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:32.842489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccbbff-691d-435c-a44d-7050d972164a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.855566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ccbbff-691d-435c-a44d-7050d972164a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.860945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242e382-5419-4f02-a68b-2490ca3ed223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.861920 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242e382-5419-4f02-a68b-2490ca3ed223 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.862791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242e382-5419-4f02-a68b-2490ca3ed223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.863244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242e382-5419-4f02-a68b-2490ca3ed223 admin admin] PATH_INFO: `/v3/projects/c100f4b0838b48aebe0b00e0639052db/users/963691a5aead4f1a9ada4fe977a2b765/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:32.884150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242e382-5419-4f02-a68b-2490ca3ed223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c100f4b0838b48aebe0b00e0639052db, user_id=963691a5aead4f1a9ada4fe977a2b765, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:32.887086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242e382-5419-4f02-a68b-2490ca3ed223 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:32.904699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1389/2750] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:32 2022] PUT /identity/v3/projects/c100f4b0838b48aebe0b00e0639052db/users/963691a5aead4f1a9ada4fe977a2b765/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:32.909209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec59b6-5c2a-4db1-9f4e-cdf443c6d42c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:32.920338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec59b6-5c2a-4db1-9f4e-cdf443c6d42c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:32.997419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece25b8-cd14-44b5-8db1-47fc05de92ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:32.998195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece25b8-cd14-44b5-8db1-47fc05de92ba admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:32.998497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece25b8-cd14-44b5-8db1-47fc05de92ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:32.998780 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece25b8-cd14-44b5-8db1-47fc05de92ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:33.010884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece25b8-cd14-44b5-8db1-47fc05de92ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:33.010884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece25b8-cd14-44b5-8db1-47fc05de92ba admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:33.014204 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1362/2751] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:32 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:33.018612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242e382-5419-4f02-a68b-2490ca3ed223 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:33.029122 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242e382-5419-4f02-a68b-2490ca3ed223 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:33.034576 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54541279-99c9-4d68-9f6c-ecf7cce26e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:33.035345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54541279-99c9-4d68-9f6c-ecf7cce26e6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:33.035633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54541279-99c9-4d68-9f6c-ecf7cce26e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:33.035884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54541279-99c9-4d68-9f6c-ecf7cce26e6d admin admin] PATH_INFO: `/v3/projects/c100f4b0838b48aebe0b00e0639052db/users/963691a5aead4f1a9ada4fe977a2b765/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:33.038061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54541279-99c9-4d68-9f6c-ecf7cce26e6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c100f4b0838b48aebe0b00e0639052db, user_id=963691a5aead4f1a9ada4fe977a2b765, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:33.040861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54541279-99c9-4d68-9f6c-ecf7cce26e6d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:33.052164 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1390/2752] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:33 2022] PUT /identity/v3/projects/c100f4b0838b48aebe0b00e0639052db/users/963691a5aead4f1a9ada4fe977a2b765/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:33.058582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece25b8-cd14-44b5-8db1-47fc05de92ba admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:33.060538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0ece25b8-cd14-44b5-8db1-47fc05de92ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:33.064753 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f272926-53ba-4669-ab9b-b650350b02aa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:33.065294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f272926-53ba-4669-ab9b-b650350b02aa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:33.065730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f272926-53ba-4669-ab9b-b650350b02aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:33.075508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece25b8-cd14-44b5-8db1-47fc05de92ba admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:33.096459 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-9f272926-53ba-4669-ab9b-b650350b02aa None None] MFA Rules not processed for user `963691a5aead4f1a9ada4fe977a2b765`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:33.097732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9f272926-53ba-4669-ab9b-b650350b02aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:33.131335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:33.139224 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:33.139898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:33.140209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:33.140848 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] PATH_INFO: `/v3/projects/f4e0d81eadb44948a3be540b96eb5db5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:33.142278 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e0d81eadb44948a3be540b96eb5db5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:33.144881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:33.174320 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1391/2753] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:33.186477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f272926-53ba-4669-ab9b-b650350b02aa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:33.189048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9f272926-53ba-4669-ab9b-b650350b02aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:33.201352 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e0d81eadb44948a3be540b96eb5db5', 'request_id': 'req-6911de86-930c-4f1a-8568-e0fedb91b957'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.207588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e0d81eadb44948a3be540b96eb5db5', 'request_id': 'req-6911de86-930c-4f1a-8568-e0fedb91b957'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.218832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e0d81eadb44948a3be540b96eb5db5', 'request_id': 'req-6911de86-930c-4f1a-8568-e0fedb91b957'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.220465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:33.238133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] The token cache is being invalidate because project f4e0d81eadb44948a3be540b96eb5db5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:33.238570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.239315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.240936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1363/2754] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:33 2022] DELETE /identity/v3/projects/f4e0d81eadb44948a3be540b96eb5db5 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:33.244874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:33.246459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:33.270317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f272926-53ba-4669-ab9b-b650350b02aa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:33.322497 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6911de86-930c-4f1a-8568-e0fedb91b957 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:33.323807 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5e35b-b1f6-4e5c-8085-ff449ef035bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:33.324936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5e35b-b1f6-4e5c-8085-ff449ef035bf service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:33.325220 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5e35b-b1f6-4e5c-8085-ff449ef035bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:33.325899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c5e35b-b1f6-4e5c-8085-ff449ef035bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:33.327798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:33.328027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-08c5e35b-b1f6-4e5c-8085-ff449ef035bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:33.328742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:33.329065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:33.329395 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] PATH_INFO: `/v3/users/b36d842174954f4f83c8cb34b07f1cd9` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:33.330856 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36d842174954f4f83c8cb34b07f1cd9)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:33.332928 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:33.380474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36d842174954f4f83c8cb34b07f1cd9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.381243 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36d842174954f4f83c8cb34b07f1cd9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.393990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36d842174954f4f83c8cb34b07f1cd9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.394183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5e35b-b1f6-4e5c-8085-ff449ef035bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:33.395609 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c5e35b-b1f6-4e5c-8085-ff449ef035bf service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:33.401597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36d842174954f4f83c8cb34b07f1cd9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.406160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1392/2755] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:33.409872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36d842174954f4f83c8cb34b07f1cd9'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.416499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1364/2756] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:33 2022] DELETE /identity/v3/users/b36d842174954f4f83c8cb34b07f1cd9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:33.694535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5e35b-b1f6-4e5c-8085-ff449ef035bf service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:33.708235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c5e35b-b1f6-4e5c-8085-ff449ef035bf service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:33.770217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:33.770870 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:33.771167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:33.771403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] PATH_INFO: `/v3/projects/0fd8da4b4be145ad88ab6aca910493a3` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:33.772545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fd8da4b4be145ad88ab6aca910493a3)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:33.774497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:33.806687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:33.820443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:33.822729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:33.833637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fd8da4b4be145ad88ab6aca910493a3', 'request_id': 'req-6dacefb4-a257-4184-b785-0ff425ce7327'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.835772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324ddd6-14b0-4060-b2b4-2330cd1062a4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:33.839748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:33.840346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fd8da4b4be145ad88ab6aca910493a3', 'request_id': 'req-6dacefb4-a257-4184-b785-0ff425ce7327'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.847082 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fd8da4b4be145ad88ab6aca910493a3', 'request_id': 'req-6dacefb4-a257-4184-b785-0ff425ce7327'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.860101 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] The token cache is being invalidate because project 0fd8da4b4be145ad88ab6aca910493a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:33.860394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.861074 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.862745 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1393/2757] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:33 2022] DELETE /identity/v3/projects/0fd8da4b4be145ad88ab6aca910493a3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:33.866881 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:33.868262 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:33.896742 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:33.897419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:33.897701 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:33.898039 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] PATH_INFO: `/v3/users/fac7aa9bd3a549acae77f72a0ca4e2db` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:33.899288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fac7aa9bd3a549acae77f72a0ca4e2db)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:33.901195 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:33.933123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacefb4-a257-4184-b785-0ff425ce7327 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:33.938529 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:33.939186 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:33.939472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:33.939752 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] PATH_INFO: `/v3/users/12c47960b52345c981aee69705ed11bf` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:33.941073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] RBAC: Authorizing `identity:delete_user(user_id=12c47960b52345c981aee69705ed11bf)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:33.943067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:33.947555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fac7aa9bd3a549acae77f72a0ca4e2db'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.948187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fac7aa9bd3a549acae77f72a0ca4e2db'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.961202 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fac7aa9bd3a549acae77f72a0ca4e2db'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.969292 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fac7aa9bd3a549acae77f72a0ca4e2db'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.978598 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-52166d66-4419-4242-9433-87bd10f7aa48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fac7aa9bd3a549acae77f72a0ca4e2db'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.984708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1365/2758] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:33 2022] DELETE /identity/v3/users/fac7aa9bd3a549acae77f72a0ca4e2db => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:33.989110 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dc5cd-8819-408d-ab67-353576f78ba6 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:33.989405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dc5cd-8819-408d-ab67-353576f78ba6 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:33.989686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77dc5cd-8819-408d-ab67-353576f78ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:33.994550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12c47960b52345c981aee69705ed11bf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:33.994550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12c47960b52345c981aee69705ed11bf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.009883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12c47960b52345c981aee69705ed11bf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.011468 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-b77dc5cd-8819-408d-ab67-353576f78ba6 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:34.014169 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b77dc5cd-8819-408d-ab67-353576f78ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.034891 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12c47960b52345c981aee69705ed11bf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.044364 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12c47960b52345c981aee69705ed11bf'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.053447 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1394/2759] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:33 2022] DELETE /identity/v3/users/12c47960b52345c981aee69705ed11bf => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:34.054322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:34.055658 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.103333 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1366/2760] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:34.108811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:34.126565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dc5cd-8819-408d-ab67-353576f78ba6 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:34.126565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b77dc5cd-8819-408d-ab67-353576f78ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.155849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9476f-b63a-4d86-a56a-52e1980e2add admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:34.163700 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:34.164688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:34.165207 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:34.166059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] PATH_INFO: `/v3/users/cd41363c5b3c4dfaba3ee905fa7242bb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:34.167375 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd41363c5b3c4dfaba3ee905fa7242bb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:34.170333 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:34.172428 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:34.220299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77dc5cd-8819-408d-ab67-353576f78ba6 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:34.249663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd41363c5b3c4dfaba3ee905fa7242bb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.250692 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd41363c5b3c4dfaba3ee905fa7242bb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.270220 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd41363c5b3c4dfaba3ee905fa7242bb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.284877 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd41363c5b3c4dfaba3ee905fa7242bb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.298342 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9035c1e7-9969-4261-a6c6-d53154280434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd41363c5b3c4dfaba3ee905fa7242bb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.300255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9659f4-b6eb-4bc0-ada7-4848d2ea47fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:34.301378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9659f4-b6eb-4bc0-ada7-4848d2ea47fe service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:34.302022 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9659f4-b6eb-4bc0-ada7-4848d2ea47fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:34.302527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9659f4-b6eb-4bc0-ada7-4848d2ea47fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:34.304748 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc9659f4-b6eb-4bc0-ada7-4848d2ea47fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.310181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1395/2761] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:34 2022] DELETE /identity/v3/users/cd41363c5b3c4dfaba3ee905fa7242bb => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:34.315645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120472-b2cc-4f42-80fe-324d176c3a48 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:34.316039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120472-b2cc-4f42-80fe-324d176c3a48 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:34.316429 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50120472-b2cc-4f42-80fe-324d176c3a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:34.320316 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9659f4-b6eb-4bc0-ada7-4848d2ea47fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:34.322337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9659f4-b6eb-4bc0-ada7-4848d2ea47fe service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:34.335033 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1367/2762] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:34.343566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-50120472-b2cc-4f42-80fe-324d176c3a48 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:34.345097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-50120472-b2cc-4f42-80fe-324d176c3a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.378574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9659f4-b6eb-4bc0-ada7-4848d2ea47fe service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:34.381732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-cc9659f4-b6eb-4bc0-ada7-4848d2ea47fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.401980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9659f4-b6eb-4bc0-ada7-4848d2ea47fe service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:34.429261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1396/2763] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:34.434523 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50120472-b2cc-4f42-80fe-324d176c3a48 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:34.440152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-50120472-b2cc-4f42-80fe-324d176c3a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.504640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:34.505551 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:34.505872 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:34.506025 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50120472-b2cc-4f42-80fe-324d176c3a48 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:34.506140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] PATH_INFO: `/v3/projects/c91559e3406944fa910f3dadd1cab92a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:34.507743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c91559e3406944fa910f3dadd1cab92a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:34.510338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:34.511689 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8bb38-44c4-4cc0-bd02-77f681eb5eab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:34.512536 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8bb38-44c4-4cc0-bd02-77f681eb5eab service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:34.513048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8bb38-44c4-4cc0-bd02-77f681eb5eab service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:34.513334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8bb38-44c4-4cc0-bd02-77f681eb5eab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:34.515648 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-81f8bb38-44c4-4cc0-bd02-77f681eb5eab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.524210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:34.594714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c91559e3406944fa910f3dadd1cab92a', 'request_id': 'req-d4211b0c-ed48-4a89-a714-1a04c0684322'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.602696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c91559e3406944fa910f3dadd1cab92a', 'request_id': 'req-d4211b0c-ed48-4a89-a714-1a04c0684322'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.611671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c91559e3406944fa910f3dadd1cab92a', 'request_id': 'req-d4211b0c-ed48-4a89-a714-1a04c0684322'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.626603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] The token cache is being invalidate because project c91559e3406944fa910f3dadd1cab92a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:34.627299 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.628219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.630542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1368/2764] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:34 2022] DELETE /identity/v3/projects/c91559e3406944fa910f3dadd1cab92a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:34.635796 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:34.638130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.660376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:34.711357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4211b0c-ed48-4a89-a714-1a04c0684322 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:34.715743 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b327055-04c9-4582-9d6d-f805200b85df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:34.716376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b327055-04c9-4582-9d6d-f805200b85df service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:34.716661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b327055-04c9-4582-9d6d-f805200b85df service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:34.716938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b327055-04c9-4582-9d6d-f805200b85df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:34.719357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b327055-04c9-4582-9d6d-f805200b85df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.731249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8bb38-44c4-4cc0-bd02-77f681eb5eab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:34.732696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8bb38-44c4-4cc0-bd02-77f681eb5eab service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:34.742943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1397/2765] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:34 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:34.748127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19955b09-d867-48f8-bb26-7a03984df597 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:34.748646 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19955b09-d867-48f8-bb26-7a03984df597 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:34.749073 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19955b09-d867-48f8-bb26-7a03984df597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:34.769142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-19955b09-d867-48f8-bb26-7a03984df597 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:34.770403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19955b09-d867-48f8-bb26-7a03984df597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.795928 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b327055-04c9-4582-9d6d-f805200b85df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:34.797995 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b327055-04c9-4582-9d6d-f805200b85df service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:34.809446 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1369/2766] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:34.810321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b327055-04c9-4582-9d6d-f805200b85df service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:34.815121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b327055-04c9-4582-9d6d-f805200b85df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.829842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b327055-04c9-4582-9d6d-f805200b85df service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:34.835798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:34.836515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:34.836890 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:34.837601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] PATH_INFO: `/v3/projects/6f323246401f468f9ac0f926c64d36c6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:34.839269 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f323246401f468f9ac0f926c64d36c6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:34.841319 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1398/2767] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:34.841905 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:34.850167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766a669-c2f5-44a2-b139-8442a34d67aa None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:34.850460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766a669-c2f5-44a2-b139-8442a34d67aa None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:34.850808 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766a669-c2f5-44a2-b139-8442a34d67aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:34.872560 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4766a669-c2f5-44a2-b139-8442a34d67aa None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:34.873153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4766a669-c2f5-44a2-b139-8442a34d67aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.882945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1399/2768] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:34.887266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766a669-c2f5-44a2-b139-8442a34d67aa None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:34.898100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766a669-c2f5-44a2-b139-8442a34d67aa None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:34.901496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f323246401f468f9ac0f926c64d36c6', 'request_id': 'req-076ddc19-29cb-4687-b807-b750385b95e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.907586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f323246401f468f9ac0f926c64d36c6', 'request_id': 'req-076ddc19-29cb-4687-b807-b750385b95e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.916226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f323246401f468f9ac0f926c64d36c6', 'request_id': 'req-076ddc19-29cb-4687-b807-b750385b95e1'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.931804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] The token cache is being invalidate because project 6f323246401f468f9ac0f926c64d36c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:34.932162 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.932981 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:34.934866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1370/2769] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:34 2022] DELETE /identity/v3/projects/6f323246401f468f9ac0f926c64d36c6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:34.938659 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:34.940304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:34.967583 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb7234-3eb6-467d-ba92-753c9d47a887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:34.968218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb7234-3eb6-467d-ba92-753c9d47a887 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:34.968504 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb7234-3eb6-467d-ba92-753c9d47a887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:34.968777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb7234-3eb6-467d-ba92-753c9d47a887 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:34.969481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb7234-3eb6-467d-ba92-753c9d47a887 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:01:34.969810 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb7234-3eb6-467d-ba92-753c9d47a887 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:34.971604 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb7234-3eb6-467d-ba92-753c9d47a887 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:34.983480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1400/2770] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 26 08:01:34.988236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb7234-3eb6-467d-ba92-753c9d47a887 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:34.990240 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-91cb7234-3eb6-467d-ba92-753c9d47a887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:35.004118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb7234-3eb6-467d-ba92-753c9d47a887 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:35.005515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:35.009282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be393251-ad3c-4d4f-81c4-ea5971fd655b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:35.010065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be393251-ad3c-4d4f-81c4-ea5971fd655b admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.010451 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be393251-ad3c-4d4f-81c4-ea5971fd655b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.010824 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be393251-ad3c-4d4f-81c4-ea5971fd655b admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.011588 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be393251-ad3c-4d4f-81c4-ea5971fd655b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:35.013589 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be393251-ad3c-4d4f-81c4-ea5971fd655b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:35.026234 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ddc19-29cb-4687-b807-b750385b95e1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:35.034569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:35.035155 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.035440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.035837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] PATH_INFO: `/v3/users/963691a5aead4f1a9ada4fe977a2b765` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.037173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=963691a5aead4f1a9ada4fe977a2b765)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:35.039281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:35.046221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1401/2771] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:34 2022] POST /identity/v3/projects => generated 375 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:35.050264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be393251-ad3c-4d4f-81c4-ea5971fd655b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:35.061105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:35.100148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963691a5aead4f1a9ada4fe977a2b765'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.100943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963691a5aead4f1a9ada4fe977a2b765'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.115106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963691a5aead4f1a9ada4fe977a2b765'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.122563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963691a5aead4f1a9ada4fe977a2b765'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.134946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963691a5aead4f1a9ada4fe977a2b765'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.142298 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1371/2772] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:34 2022] DELETE /identity/v3/users/963691a5aead4f1a9ada4fe977a2b765 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:35.150678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be393251-ad3c-4d4f-81c4-ea5971fd655b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:35.216909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03a12e-57b8-464a-a461-0b19ff675b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:35.217633 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03a12e-57b8-464a-a461-0b19ff675b86 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.217945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03a12e-57b8-464a-a461-0b19ff675b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.219103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03a12e-57b8-464a-a461-0b19ff675b86 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.219103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03a12e-57b8-464a-a461-0b19ff675b86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:35.220888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b03a12e-57b8-464a-a461-0b19ff675b86 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:35.250626 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1402/2773] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:35 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:35.255173 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:35.256889 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:35.261517 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03a12e-57b8-464a-a461-0b19ff675b86 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:35.263482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5b03a12e-57b8-464a-a461-0b19ff675b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:35.273547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038926d-7a9e-4a3f-a1dd-aeece379c1c0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:35.280226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22c335-beff-4186-88e6-8da329a25304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:35.280868 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22c335-beff-4186-88e6-8da329a25304 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.281219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22c335-beff-4186-88e6-8da329a25304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.281361 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03a12e-57b8-464a-a461-0b19ff675b86 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:35.281474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22c335-beff-4186-88e6-8da329a25304 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.282403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22c335-beff-4186-88e6-8da329a25304 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:35.285018 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22c335-beff-4186-88e6-8da329a25304 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:35.286771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:35.287739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.288070 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.288413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] PATH_INFO: `/v3/projects/7c22fc1cc86841b3bd226ab15f8b6568` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.289759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c22fc1cc86841b3bd226ab15f8b6568)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:35.292563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:35.302061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1372/2774] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:35.303875 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22c335-beff-4186-88e6-8da329a25304 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:35.319606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22c335-beff-4186-88e6-8da329a25304 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:35.326439 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac792b-d1fe-48eb-a1e0-4d6cd75b7883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:35.327184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac792b-d1fe-48eb-a1e0-4d6cd75b7883 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.327768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac792b-d1fe-48eb-a1e0-4d6cd75b7883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.328125 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac792b-d1fe-48eb-a1e0-4d6cd75b7883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.328917 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac792b-d1fe-48eb-a1e0-4d6cd75b7883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:35.331077 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac792b-d1fe-48eb-a1e0-4d6cd75b7883 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:35.342903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1373/2775] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:35.347278 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac792b-d1fe-48eb-a1e0-4d6cd75b7883 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:35.349406 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c22fc1cc86841b3bd226ab15f8b6568', 'request_id': 'req-b06d4de9-5a6b-4299-8a2d-8acefd578978'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.357698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c22fc1cc86841b3bd226ab15f8b6568', 'request_id': 'req-b06d4de9-5a6b-4299-8a2d-8acefd578978'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.358893 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac792b-d1fe-48eb-a1e0-4d6cd75b7883 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:35.363900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c22fc1cc86841b3bd226ab15f8b6568', 'request_id': 'req-b06d4de9-5a6b-4299-8a2d-8acefd578978'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.381473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] The token cache is being invalidate because project 7c22fc1cc86841b3bd226ab15f8b6568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:35.381832 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.382486 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.387245 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1403/2776] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:35 2022] DELETE /identity/v3/projects/7c22fc1cc86841b3bd226ab15f8b6568 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:35.388511 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:35.390448 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:35.428946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:35.439574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b5f54e-3d38-4224-a5f9-a4eade9665a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:35.439574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5f54e-3d38-4224-a5f9-a4eade9665a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.439574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5f54e-3d38-4224-a5f9-a4eade9665a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.439574 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b5f54e-3d38-4224-a5f9-a4eade9665a8 admin admin] PATH_INFO: `/v3/projects/51d5adeabc614cf48df9369517491baf/users/23c72c39761642f9982efefe6e074906/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.466913 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5f54e-3d38-4224-a5f9-a4eade9665a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d5adeabc614cf48df9369517491baf, user_id=23c72c39761642f9982efefe6e074906, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:35.481708 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b5f54e-3d38-4224-a5f9-a4eade9665a8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:35.482996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06d4de9-5a6b-4299-8a2d-8acefd578978 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:35.491227 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:35.492191 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.493960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.494564 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] PATH_INFO: `/v3/users/841aa3257ceb4d379e00ca51e881d379` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.497005 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=841aa3257ceb4d379e00ca51e881d379)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:35.501970 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:35.504434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1374/2777] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:35 2022] PUT /identity/v3/projects/51d5adeabc614cf48df9369517491baf/users/23c72c39761642f9982efefe6e074906/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:35.515769 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c565e80-438d-4ecc-b4e6-5f56de74b7e7 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.516196 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c565e80-438d-4ecc-b4e6-5f56de74b7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.516486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c565e80-438d-4ecc-b4e6-5f56de74b7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.561791 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-7c565e80-438d-4ecc-b4e6-5f56de74b7e7 None None] MFA Rules not processed for user `23c72c39761642f9982efefe6e074906`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:35.563898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c565e80-438d-4ecc-b4e6-5f56de74b7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:35.575384 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '841aa3257ceb4d379e00ca51e881d379'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.576440 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '841aa3257ceb4d379e00ca51e881d379'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.595280 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:35.597304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '841aa3257ceb4d379e00ca51e881d379'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.605811 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '841aa3257ceb4d379e00ca51e881d379'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.624058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '841aa3257ceb4d379e00ca51e881d379'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.638581 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1404/2778] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:35 2022] DELETE /identity/v3/users/841aa3257ceb4d379e00ca51e881d379 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:35.658780 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:35.661016 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:35.668107 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1375/2779] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:35.690980 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c565e80-438d-4ecc-b4e6-5f56de74b7e7 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:35.695413 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7c565e80-438d-4ecc-b4e6-5f56de74b7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:35.759832 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02625b00-4201-4c84-b286-1ceb5e51bb78 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:35.765570 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:35.766359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.767236 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.767520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] PATH_INFO: `/v3/projects/c100f4b0838b48aebe0b00e0639052db` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.768771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c100f4b0838b48aebe0b00e0639052db)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:35.770777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:35.788979 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c565e80-438d-4ecc-b4e6-5f56de74b7e7 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:35.831346 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c100f4b0838b48aebe0b00e0639052db', 'request_id': 'req-587006fe-fc39-4106-a01d-9637941eff00'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.832724 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:35.857510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c100f4b0838b48aebe0b00e0639052db', 'request_id': 'req-587006fe-fc39-4106-a01d-9637941eff00'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.866355 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3f59c-2c34-4f4a-81ca-b4771036d40c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c6943686c843e588df14f7c6dffb8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:35.866965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3f59c-2c34-4f4a-81ca-b4771036d40c service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.867240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3f59c-2c34-4f4a-81ca-b4771036d40c service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.867513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3f59c-2c34-4f4a-81ca-b4771036d40c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.869381 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c100f4b0838b48aebe0b00e0639052db', 'request_id': 'req-587006fe-fc39-4106-a01d-9637941eff00'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.894527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] The token cache is being invalidate because project c100f4b0838b48aebe0b00e0639052db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:35.894995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.895885 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-587006fe-fc39-4106-a01d-9637941eff00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:35.901833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1405/2780] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:35 2022] DELETE /identity/v3/projects/c100f4b0838b48aebe0b00e0639052db => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:35.906785 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eb1f9-f50f-4fd1-b892-1474e6411cd8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:35.907477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eb1f9-f50f-4fd1-b892-1474e6411cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:35.907477 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13eb1f9-f50f-4fd1-b892-1474e6411cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:35.934762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-d13eb1f9-f50f-4fd1-b892-1474e6411cd8 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:35.936480 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-d13eb1f9-f50f-4fd1-b892-1474e6411cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:35.948912 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3f59c-2c34-4f4a-81ca-b4771036d40c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:35.950770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3f59c-2c34-4f4a-81ca-b4771036d40c service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:35.961296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1376/2781] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:35.985082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3f59c-2c34-4f4a-81ca-b4771036d40c service glance] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:35.986673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-1cd3f59c-2c34-4f4a-81ca-b4771036d40c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:36.001005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3f59c-2c34-4f4a-81ca-b4771036d40c service glance] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.036095 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=100 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:36.047421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1406/2782] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:36.058801 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864b5b4-d477-4307-b6ef-420f9d8ddb64 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.059267 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864b5b4-d477-4307-b6ef-420f9d8ddb64 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.059708 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864b5b4-d477-4307-b6ef-420f9d8ddb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.098348 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.099281 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-8864b5b4-d477-4307-b6ef-420f9d8ddb64 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:36.099548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.099983 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.100899 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-8864b5b4-d477-4307-b6ef-420f9d8ddb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:36.101935 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] PATH_INFO: `/v3/projects/db087410b4e249b883ef3352de322c17` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.104359 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db087410b4e249b883ef3352de322c17)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.107388 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.117302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1407/2783] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:36.119084 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864b5b4-d477-4307-b6ef-420f9d8ddb64 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.131221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864b5b4-d477-4307-b6ef-420f9d8ddb64 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.146647 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de437b-bcb6-44f5-9a0a-d34fb92c8b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.147608 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de437b-bcb6-44f5-9a0a-d34fb92c8b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.148189 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de437b-bcb6-44f5-9a0a-d34fb92c8b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.148531 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45de437b-bcb6-44f5-9a0a-d34fb92c8b31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.149325 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de437b-bcb6-44f5-9a0a-d34fb92c8b31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:01:36.149683 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de437b-bcb6-44f5-9a0a-d34fb92c8b31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.151497 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45de437b-bcb6-44f5-9a0a-d34fb92c8b31 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.169887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1408/2784] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:01:36.171142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de437b-bcb6-44f5-9a0a-d34fb92c8b31 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.180171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db087410b4e249b883ef3352de322c17', 'request_id': 'req-0d42afd8-078c-49d1-b29a-fc4d50516a04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:36.183154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45de437b-bcb6-44f5-9a0a-d34fb92c8b31 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.190076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db087410b4e249b883ef3352de322c17', 'request_id': 'req-0d42afd8-078c-49d1-b29a-fc4d50516a04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:36.201837 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db087410b4e249b883ef3352de322c17', 'request_id': 'req-0d42afd8-078c-49d1-b29a-fc4d50516a04'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:36.216543 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] The token cache is being invalidate because project db087410b4e249b883ef3352de322c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:36.216899 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:36.217835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-0d42afd8-078c-49d1-b29a-fc4d50516a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:36.219811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1377/2785] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:35 2022] DELETE /identity/v3/projects/db087410b4e249b883ef3352de322c17 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:36.225576 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0a783-1aa7-4728-8d88-88c73c95d4af None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.225971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0a783-1aa7-4728-8d88-88c73c95d4af None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.226303 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0a783-1aa7-4728-8d88-88c73c95d4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.247130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-67f0a783-1aa7-4728-8d88-88c73c95d4af None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:36.248454 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-67f0a783-1aa7-4728-8d88-88c73c95d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:36.276925 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6fbc5-843d-478d-9a29-14834d117084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.277984 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6fbc5-843d-478d-9a29-14834d117084 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.278261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6fbc5-843d-478d-9a29-14834d117084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.278540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff6fbc5-843d-478d-9a29-14834d117084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.279918 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6fbc5-843d-478d-9a29-14834d117084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.281985 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff6fbc5-843d-478d-9a29-14834d117084 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.325312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1409/2786] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/projects => generated 365 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:36.330347 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6fbc5-843d-478d-9a29-14834d117084 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.332302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2ff6fbc5-843d-478d-9a29-14834d117084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:36.352668 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1378/2787] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:36.362270 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed0e8-571a-4b6a-8dda-bade40b05b5d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.362558 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed0e8-571a-4b6a-8dda-bade40b05b5d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.362835 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ed0e8-571a-4b6a-8dda-bade40b05b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.384306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:36.401926 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-fc4ed0e8-571a-4b6a-8dda-bade40b05b5d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:36.403187 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-fc4ed0e8-571a-4b6a-8dda-bade40b05b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:36.416399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff6fbc5-843d-478d-9a29-14834d117084 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.418121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1379/2788] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:36.429260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed0e8-571a-4b6a-8dda-bade40b05b5d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.430792 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a00fa4-5736-4281-a417-5c97bb2ba027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.431456 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00fa4-5736-4281-a417-5c97bb2ba027 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.431742 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00fa4-5736-4281-a417-5c97bb2ba027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.432324 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a00fa4-5736-4281-a417-5c97bb2ba027 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.433777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a00fa4-5736-4281-a417-5c97bb2ba027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.437856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a00fa4-5736-4281-a417-5c97bb2ba027 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.438151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ed0e8-571a-4b6a-8dda-bade40b05b5d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.449075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc447f68-4dc0-4f30-8141-c68aa96bd0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.449770 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc447f68-4dc0-4f30-8141-c68aa96bd0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.450057 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc447f68-4dc0-4f30-8141-c68aa96bd0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.450353 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc447f68-4dc0-4f30-8141-c68aa96bd0f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.451059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc447f68-4dc0-4f30-8141-c68aa96bd0f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:01:36.451354 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc447f68-4dc0-4f30-8141-c68aa96bd0f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.453439 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc447f68-4dc0-4f30-8141-c68aa96bd0f1 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.470477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1380/2789] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:01:36.475306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc447f68-4dc0-4f30-8141-c68aa96bd0f1 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.477046 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1410/2790] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/users => generated 283 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:36.480995 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a00fa4-5736-4281-a417-5c97bb2ba027 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.493452 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a00fa4-5736-4281-a417-5c97bb2ba027 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.499276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50093cc8-0416-4fd3-92ba-294cbd2b236e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.499917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50093cc8-0416-4fd3-92ba-294cbd2b236e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.500203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50093cc8-0416-4fd3-92ba-294cbd2b236e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.500537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50093cc8-0416-4fd3-92ba-294cbd2b236e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.501176 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50093cc8-0416-4fd3-92ba-294cbd2b236e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.502878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50093cc8-0416-4fd3-92ba-294cbd2b236e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.506445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc447f68-4dc0-4f30-8141-c68aa96bd0f1 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.522905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1411/2791] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:36.523065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b16244-4c01-426c-bf3b-bbe3463327d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.523065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b16244-4c01-426c-bf3b-bbe3463327d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.523065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b16244-4c01-426c-bf3b-bbe3463327d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.523065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b16244-4c01-426c-bf3b-bbe3463327d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.523510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b16244-4c01-426c-bf3b-bbe3463327d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.523510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b16244-4c01-426c-bf3b-bbe3463327d4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.527444 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50093cc8-0416-4fd3-92ba-294cbd2b236e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.557060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50093cc8-0416-4fd3-92ba-294cbd2b236e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.567505 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d376-fedb-44ac-b7f9-89d1b4383f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.568152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d376-fedb-44ac-b7f9-89d1b4383f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.568437 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d376-fedb-44ac-b7f9-89d1b4383f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.568715 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f4d376-fedb-44ac-b7f9-89d1b4383f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.570946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4d376-fedb-44ac-b7f9-89d1b4383f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.572562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f4d376-fedb-44ac-b7f9-89d1b4383f0e admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.574952 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1381/2792] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:36.585461 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b16244-4c01-426c-bf3b-bbe3463327d4 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.593887 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1412/2793] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:36.595341 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d376-fedb-44ac-b7f9-89d1b4383f0e admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.607739 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f4d376-fedb-44ac-b7f9-89d1b4383f0e admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.611410 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:36.670079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b16244-4c01-426c-bf3b-bbe3463327d4 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.677336 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928e828-8d84-41be-852d-af3be3519c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.677517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928e828-8d84-41be-852d-af3be3519c02 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.677814 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928e828-8d84-41be-852d-af3be3519c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.678085 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928e828-8d84-41be-852d-af3be3519c02 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.678758 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928e828-8d84-41be-852d-af3be3519c02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.681006 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928e828-8d84-41be-852d-af3be3519c02 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.693396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a731ed-c2e9-4b34-9c1f-f484786d56b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.694204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a731ed-c2e9-4b34-9c1f-f484786d56b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.694543 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a731ed-c2e9-4b34-9c1f-f484786d56b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.694854 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a731ed-c2e9-4b34-9c1f-f484786d56b4 admin admin] PATH_INFO: `/v3/projects/fee28ced1ed34219b4829d1ff3ca3f20/users/20046700233d4091bb343d391a1ac6da/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.703702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1382/2794] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:36.707900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928e828-8d84-41be-852d-af3be3519c02 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.715316 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a731ed-c2e9-4b34-9c1f-f484786d56b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee28ced1ed34219b4829d1ff3ca3f20, user_id=20046700233d4091bb343d391a1ac6da, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.718011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a731ed-c2e9-4b34-9c1f-f484786d56b4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.720380 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928e828-8d84-41be-852d-af3be3519c02 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.725619 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fd9e1-4809-42b0-9324-39abf8ee11ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.726391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fd9e1-4809-42b0-9324-39abf8ee11ff admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.726740 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fd9e1-4809-42b0-9324-39abf8ee11ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.727067 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3fd9e1-4809-42b0-9324-39abf8ee11ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.727951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fd9e1-4809-42b0-9324-39abf8ee11ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.729612 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3fd9e1-4809-42b0-9324-39abf8ee11ff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.731884 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1413/2795] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:36 2022] PUT /identity/v3/projects/fee28ced1ed34219b4829d1ff3ca3f20/users/20046700233d4091bb343d391a1ac6da/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:36.744037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66e1f3-0c81-43e6-8693-02a9c3d4bc2f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.744398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66e1f3-0c81-43e6-8693-02a9c3d4bc2f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.744757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66e1f3-0c81-43e6-8693-02a9c3d4bc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.746009 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1383/2796] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:36.746881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fd9e1-4809-42b0-9324-39abf8ee11ff admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.759252 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3fd9e1-4809-42b0-9324-39abf8ee11ff admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.774803 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-3e66e1f3-0c81-43e6-8693-02a9c3d4bc2f None None] MFA Rules not processed for user `20046700233d4091bb343d391a1ac6da`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:36.776039 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3e66e1f3-0c81-43e6-8693-02a9c3d4bc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:36.855093 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36144875-2d75-4e69-a77a-b792e5632038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.856218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36144875-2d75-4e69-a77a-b792e5632038 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.856888 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36144875-2d75-4e69-a77a-b792e5632038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.857537 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36144875-2d75-4e69-a77a-b792e5632038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.858523 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36144875-2d75-4e69-a77a-b792e5632038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.860463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36144875-2d75-4e69-a77a-b792e5632038 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.861378 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1414/2797] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:36.862470 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66e1f3-0c81-43e6-8693-02a9c3d4bc2f None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.874892 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66e1f3-0c81-43e6-8693-02a9c3d4bc2f None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.881317 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a1d2a-2cea-43ac-b238-ad51feb3a4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.881658 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1384/2798] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:36 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:36.882697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a1d2a-2cea-43ac-b238-ad51feb3a4da admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.883195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a1d2a-2cea-43ac-b238-ad51feb3a4da admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.883684 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a1d2a-2cea-43ac-b238-ad51feb3a4da admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.884550 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a1d2a-2cea-43ac-b238-ad51feb3a4da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.886054 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36144875-2d75-4e69-a77a-b792e5632038 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.887028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a1d2a-2cea-43ac-b238-ad51feb3a4da admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.897804 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36144875-2d75-4e69-a77a-b792e5632038 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:36.917051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d984b63-145f-44dc-974f-f7efff516bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:36.917842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984b63-145f-44dc-974f-f7efff516bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.918937 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984b63-145f-44dc-974f-f7efff516bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.918937 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d984b63-145f-44dc-974f-f7efff516bd4 admin admin] PATH_INFO: `/v3/projects/91d9cbced64e4f73b37bc20c66579fc2/users/77a7ea623b2a463092fdb1ad73ae6ece/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:36.932718 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1415/2799] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:36.933900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a1d2a-2cea-43ac-b238-ad51feb3a4da admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:36.944376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d984b63-145f-44dc-974f-f7efff516bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d9cbced64e4f73b37bc20c66579fc2, user_id=77a7ea623b2a463092fdb1ad73ae6ece, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:36.947879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d984b63-145f-44dc-974f-f7efff516bd4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:36.955235 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:36.979112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1385/2800] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:36 2022] PUT /identity/v3/projects/91d9cbced64e4f73b37bc20c66579fc2/users/77a7ea623b2a463092fdb1ad73ae6ece/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:36.984503 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cb748-2a4b-4b4e-af73-e35e45417134 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:36.987759 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cb748-2a4b-4b4e-af73-e35e45417134 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:36.988716 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666cb748-2a4b-4b4e-af73-e35e45417134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.030954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-666cb748-2a4b-4b4e-af73-e35e45417134 None None] MFA Rules not processed for user `77a7ea623b2a463092fdb1ad73ae6ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:37.032842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-666cb748-2a4b-4b4e-af73-e35e45417134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:37.079840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a1d2a-2cea-43ac-b238-ad51feb3a4da admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.143858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:37.154477 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1386/2801] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:37.164074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cb748-2a4b-4b4e-af73-e35e45417134 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:37.178968 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3714dd-dd50-4378-85d6-b5dbec55384c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.180076 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3714dd-dd50-4378-85d6-b5dbec55384c admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.181290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3714dd-dd50-4378-85d6-b5dbec55384c admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.181634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3714dd-dd50-4378-85d6-b5dbec55384c admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.183261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3714dd-dd50-4378-85d6-b5dbec55384c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:37.186738 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3714dd-dd50-4378-85d6-b5dbec55384c admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:37.225655 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1416/2802] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:36 2022] POST /identity/v3/users => generated 288 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:37.227218 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3714dd-dd50-4378-85d6-b5dbec55384c admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:37.230264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3a3714dd-dd50-4378-85d6-b5dbec55384c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:37.263891 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666cb748-2a4b-4b4e-af73-e35e45417134 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.272915 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da638379-fcde-48a1-ae49-e8ca675315b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.273850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da638379-fcde-48a1-ae49-e8ca675315b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.274257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da638379-fcde-48a1-ae49-e8ca675315b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.275010 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da638379-fcde-48a1-ae49-e8ca675315b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.275728 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da638379-fcde-48a1-ae49-e8ca675315b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:37.278988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da638379-fcde-48a1-ae49-e8ca675315b2 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:37.321357 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1387/2803] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:37 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:37.322520 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da638379-fcde-48a1-ae49-e8ca675315b2 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:37.323993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3714dd-dd50-4378-85d6-b5dbec55384c admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.362244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:37.403713 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf5709-012c-482f-995f-d1217b829d37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.404653 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf5709-012c-482f-995f-d1217b829d37 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.405154 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf5709-012c-482f-995f-d1217b829d37 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.406040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bf5709-012c-482f-995f-d1217b829d37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.410388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-07bf5709-012c-482f-995f-d1217b829d37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:37.423428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf5709-012c-482f-995f-d1217b829d37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:37.425336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bf5709-012c-482f-995f-d1217b829d37 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:37.437155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1417/2804] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:37.438077 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf5709-012c-482f-995f-d1217b829d37 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:37.479512 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da638379-fcde-48a1-ae49-e8ca675315b2 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.496552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21809b75-dbbf-47ef-b5f7-f013dab20751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.497268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21809b75-dbbf-47ef-b5f7-f013dab20751 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.497760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21809b75-dbbf-47ef-b5f7-f013dab20751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.498059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21809b75-dbbf-47ef-b5f7-f013dab20751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.499159 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21809b75-dbbf-47ef-b5f7-f013dab20751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:37.500384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21809b75-dbbf-47ef-b5f7-f013dab20751 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:37.516521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1388/2805] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:37.520908 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21809b75-dbbf-47ef-b5f7-f013dab20751 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:37.531104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21809b75-dbbf-47ef-b5f7-f013dab20751 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.536894 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bf5709-012c-482f-995f-d1217b829d37 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.543212 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d24f42-5618-46c2-8e1b-67cce5280da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.543989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d24f42-5618-46c2-8e1b-67cce5280da6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.544414 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d24f42-5618-46c2-8e1b-67cce5280da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.544709 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d24f42-5618-46c2-8e1b-67cce5280da6 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.545622 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d24f42-5618-46c2-8e1b-67cce5280da6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:37.547573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d24f42-5618-46c2-8e1b-67cce5280da6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:37.551310 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7ec44-57e0-435d-85f4-6905e7e7f805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.552014 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7ec44-57e0-435d-85f4-6905e7e7f805 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.552417 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7ec44-57e0-435d-85f4-6905e7e7f805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.552588 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7ec44-57e0-435d-85f4-6905e7e7f805 admin admin] PATH_INFO: `/v3/projects/fd0386f1081147bdb93cbd78ac8dab64/users/3034bfa2aef6400f82127799f0e6bdcb/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.576989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7ec44-57e0-435d-85f4-6905e7e7f805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0386f1081147bdb93cbd78ac8dab64, user_id=3034bfa2aef6400f82127799f0e6bdcb, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:37.586783 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7ec44-57e0-435d-85f4-6905e7e7f805 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:37.591507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1418/2806] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:37 2022] POST /identity/v3/users => generated 288 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:37.595698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d24f42-5618-46c2-8e1b-67cce5280da6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:37.624155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d24f42-5618-46c2-8e1b-67cce5280da6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.627281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1389/2807] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:37 2022] PUT /identity/v3/projects/fd0386f1081147bdb93cbd78ac8dab64/users/3034bfa2aef6400f82127799f0e6bdcb/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:37.631329 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7ec44-57e0-435d-85f4-6905e7e7f805 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:37.716681 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:37.724830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7ec44-57e0-435d-85f4-6905e7e7f805 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.734900 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea078040-d23b-4bc4-a086-13ffd4a6fbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.735649 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078040-d23b-4bc4-a086-13ffd4a6fbff admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.737389 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078040-d23b-4bc4-a086-13ffd4a6fbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.738056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078040-d23b-4bc4-a086-13ffd4a6fbff admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.738898 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea078040-d23b-4bc4-a086-13ffd4a6fbff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:37.740972 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea078040-d23b-4bc4-a086-13ffd4a6fbff admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:37.745211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afae4c6-e4be-41a9-ad3a-d91b4974f159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.745997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afae4c6-e4be-41a9-ad3a-d91b4974f159 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.746399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afae4c6-e4be-41a9-ad3a-d91b4974f159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.746775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afae4c6-e4be-41a9-ad3a-d91b4974f159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.747563 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afae4c6-e4be-41a9-ad3a-d91b4974f159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:37.749698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afae4c6-e4be-41a9-ad3a-d91b4974f159 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:37.754784 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1390/2808] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:37.759209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea078040-d23b-4bc4-a086-13ffd4a6fbff admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:37.774989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea078040-d23b-4bc4-a086-13ffd4a6fbff admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.778616 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1419/2809] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:37.780167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afae4c6-e4be-41a9-ad3a-d91b4974f159 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:37.780541 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbe23e6-3e11-4833-a146-0ea5a15602d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.781257 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe23e6-3e11-4833-a146-0ea5a15602d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.781546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe23e6-3e11-4833-a146-0ea5a15602d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.781949 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbe23e6-3e11-4833-a146-0ea5a15602d7 admin admin] PATH_INFO: `/v3/projects/fd0386f1081147bdb93cbd78ac8dab64/users/3034bfa2aef6400f82127799f0e6bdcb/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.785230 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe23e6-3e11-4833-a146-0ea5a15602d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0386f1081147bdb93cbd78ac8dab64, user_id=3034bfa2aef6400f82127799f0e6bdcb, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:37.790032 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbe23e6-3e11-4833-a146-0ea5a15602d7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:37.793552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afae4c6-e4be-41a9-ad3a-d91b4974f159 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.799939 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e3a0d-f4c0-4164-ab0a-256cb3f1cdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.800679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e3a0d-f4c0-4164-ab0a-256cb3f1cdd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.801028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e3a0d-f4c0-4164-ab0a-256cb3f1cdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.801695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e3a0d-f4c0-4164-ab0a-256cb3f1cdd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.802309 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e3a0d-f4c0-4164-ab0a-256cb3f1cdd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:37.804042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820e3a0d-f4c0-4164-ab0a-256cb3f1cdd6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:37.805117 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1391/2810] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:37 2022] PUT /identity/v3/projects/fd0386f1081147bdb93cbd78ac8dab64/users/3034bfa2aef6400f82127799f0e6bdcb/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:37.815660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dffd17-5692-4eef-b96b-4969f227fce5 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.815968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dffd17-5692-4eef-b96b-4969f227fce5 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.816219 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dffd17-5692-4eef-b96b-4969f227fce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.837976 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1420/2811] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:37 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:37.841139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e3a0d-f4c0-4164-ab0a-256cb3f1cdd6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:37.861777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e3a0d-f4c0-4164-ab0a-256cb3f1cdd6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:37.891376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-40dffd17-5692-4eef-b96b-4969f227fce5 None None] MFA Rules not processed for user `3034bfa2aef6400f82127799f0e6bdcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:37.893078 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-40dffd17-5692-4eef-b96b-4969f227fce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:37.923656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:37.976035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfd503-69cc-452f-8e3d-c71b0989c474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:37.976758 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfd503-69cc-452f-8e3d-c71b0989c474 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:37.977129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfd503-69cc-452f-8e3d-c71b0989c474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:37.977595 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfd503-69cc-452f-8e3d-c71b0989c474 admin admin] PATH_INFO: `/v3/projects/be7ea3c46f44445b8def4edfdfd2ba17/users/07a53ad1569e4fa5bc3ce9e315f54acd/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:37.998637 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfd503-69cc-452f-8e3d-c71b0989c474 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be7ea3c46f44445b8def4edfdfd2ba17, user_id=07a53ad1569e4fa5bc3ce9e315f54acd, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:38.001338 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfd503-69cc-452f-8e3d-c71b0989c474 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:38.015775 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1392/2812] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:37 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:38.019431 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1421/2813] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:37 2022] PUT /identity/v3/projects/be7ea3c46f44445b8def4edfdfd2ba17/users/07a53ad1569e4fa5bc3ce9e315f54acd/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:38.031191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a60cdc-3ad4-4175-b01c-5a1bb1a6f7eb None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:38.031191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a60cdc-3ad4-4175-b01c-5a1bb1a6f7eb None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:38.031191 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a60cdc-3ad4-4175-b01c-5a1bb1a6f7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:38.041114 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfd503-69cc-452f-8e3d-c71b0989c474 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:38.043257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e0cfd503-69cc-452f-8e3d-c71b0989c474 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:38.075692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-69a60cdc-3ad4-4175-b01c-5a1bb1a6f7eb None None] MFA Rules not processed for user `07a53ad1569e4fa5bc3ce9e315f54acd`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:38.076967 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-69a60cdc-3ad4-4175-b01c-5a1bb1a6f7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:38.150261 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfd503-69cc-452f-8e3d-c71b0989c474 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:38.158749 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fd001-f1fb-4b2b-ab98-4a7b090371ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:38.159875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fd001-f1fb-4b2b-ab98-4a7b090371ef service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:38.160156 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fd001-f1fb-4b2b-ab98-4a7b090371ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:38.160693 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3fd001-f1fb-4b2b-ab98-4a7b090371ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:38.179030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1393/2814] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:38.183431 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a60cdc-3ad4-4175-b01c-5a1bb1a6f7eb None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:38.261364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:38.262465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fd001-f1fb-4b2b-ab98-4a7b090371ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:38.267298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3fd001-f1fb-4b2b-ab98-4a7b090371ef service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:38.283889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1422/2815] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:38.287940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a60cdc-3ad4-4175-b01c-5a1bb1a6f7eb None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:38.297238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c46f6-8afd-4308-85a5-c67056cae412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:38.298142 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c46f6-8afd-4308-85a5-c67056cae412 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:38.299331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c46f6-8afd-4308-85a5-c67056cae412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:38.299684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c46f6-8afd-4308-85a5-c67056cae412 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:38.300452 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c46f6-8afd-4308-85a5-c67056cae412 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:38.302487 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273c46f6-8afd-4308-85a5-c67056cae412 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:38.385798 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1394/2816] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:38 2022] POST /identity/v3/projects => generated 377 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:38.389975 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fd001-f1fb-4b2b-ab98-4a7b090371ef service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:38.406459 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c46f6-8afd-4308-85a5-c67056cae412 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:38.407623 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3fd001-f1fb-4b2b-ab98-4a7b090371ef service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:38.481048 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3790f-56e9-4fd7-9e4a-44c145b9d2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:38.481913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3790f-56e9-4fd7-9e4a-44c145b9d2ca admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:38.482312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3790f-56e9-4fd7-9e4a-44c145b9d2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:38.482840 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3790f-56e9-4fd7-9e4a-44c145b9d2ca admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:38.483603 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3790f-56e9-4fd7-9e4a-44c145b9d2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:38.485707 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf3790f-56e9-4fd7-9e4a-44c145b9d2ca admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:38.508715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c46f6-8afd-4308-85a5-c67056cae412 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:38.512460 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:38.515081 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbabdf-9adc-4097-85ac-70c621a457ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:38.515883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbabdf-9adc-4097-85ac-70c621a457ee service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:38.516206 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbabdf-9adc-4097-85ac-70c621a457ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:38.516489 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbabdf-9adc-4097-85ac-70c621a457ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:38.520577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1423/2817] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:38 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:38.526921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3790f-56e9-4fd7-9e4a-44c145b9d2ca admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:38.538998 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf3790f-56e9-4fd7-9e4a-44c145b9d2ca admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:38.550433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a958d46-dbe6-4328-b884-03d5380d4ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:38.551348 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a958d46-dbe6-4328-b884-03d5380d4ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:38.551886 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a958d46-dbe6-4328-b884-03d5380d4ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:38.552166 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a958d46-dbe6-4328-b884-03d5380d4ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:38.552889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a958d46-dbe6-4328-b884-03d5380d4ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:38.554687 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a958d46-dbe6-4328-b884-03d5380d4ba6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:38.568081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1424/2818] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:38.573152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a958d46-dbe6-4328-b884-03d5380d4ba6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:38.595559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a958d46-dbe6-4328-b884-03d5380d4ba6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:38.601555 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbabdf-9adc-4097-85ac-70c621a457ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:38.603104 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211c4dd-dbba-4f4f-8156-da33a9665c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:38.603288 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecbabdf-9adc-4097-85ac-70c621a457ee service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:38.604033 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211c4dd-dbba-4f4f-8156-da33a9665c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:38.604422 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211c4dd-dbba-4f4f-8156-da33a9665c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:38.604760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211c4dd-dbba-4f4f-8156-da33a9665c60 admin admin] PATH_INFO: `/v3/projects/e3d3b7245bc7468c9fd0f7ee8f6fe34a/users/bea1cc64dbdc4e0cb38dac13097c8103/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:38.614068 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1395/2819] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:01:38.623936 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211c4dd-dbba-4f4f-8156-da33a9665c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d3b7245bc7468c9fd0f7ee8f6fe34a, user_id=bea1cc64dbdc4e0cb38dac13097c8103, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:38.627198 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4211c4dd-dbba-4f4f-8156-da33a9665c60 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:38.653264 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1425/2820] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:38 2022] PUT /identity/v3/projects/e3d3b7245bc7468c9fd0f7ee8f6fe34a/users/bea1cc64dbdc4e0cb38dac13097c8103/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:38.653992 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbabdf-9adc-4097-85ac-70c621a457ee service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:38.665024 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbabdf-9adc-4097-85ac-70c621a457ee service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:38.752929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d518fb-8698-4265-b418-792ae5ca81f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:38.753842 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d518fb-8698-4265-b418-792ae5ca81f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:38.754133 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d518fb-8698-4265-b418-792ae5ca81f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:38.754417 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d518fb-8698-4265-b418-792ae5ca81f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:38.756631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d518fb-8698-4265-b418-792ae5ca81f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:38.758272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d518fb-8698-4265-b418-792ae5ca81f6 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:38.772712 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1396/2821] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:38 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:38.774398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211c4dd-dbba-4f4f-8156-da33a9665c60 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:38.795717 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211c4dd-dbba-4f4f-8156-da33a9665c60 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:38.803044 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd31a49-f1bc-45d4-a119-8391b6e6569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:38.803949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd31a49-f1bc-45d4-a119-8391b6e6569d admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:38.804358 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd31a49-f1bc-45d4-a119-8391b6e6569d admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:38.805170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd31a49-f1bc-45d4-a119-8391b6e6569d admin admin] PATH_INFO: `/v3/projects/e3d3b7245bc7468c9fd0f7ee8f6fe34a/users/bea1cc64dbdc4e0cb38dac13097c8103/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:38.807510 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd31a49-f1bc-45d4-a119-8391b6e6569d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d3b7245bc7468c9fd0f7ee8f6fe34a, user_id=bea1cc64dbdc4e0cb38dac13097c8103, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:38.810305 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd31a49-f1bc-45d4-a119-8391b6e6569d admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:38.831809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1426/2822] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:38 2022] PUT /identity/v3/projects/e3d3b7245bc7468c9fd0f7ee8f6fe34a/users/bea1cc64dbdc4e0cb38dac13097c8103/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:38.841929 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc79c8-e9e3-4517-8b65-cd458081aea8 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:38.842226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc79c8-e9e3-4517-8b65-cd458081aea8 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:38.842499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc79c8-e9e3-4517-8b65-cd458081aea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:38.872838 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-f3bc79c8-e9e3-4517-8b65-cd458081aea8 None None] MFA Rules not processed for user `bea1cc64dbdc4e0cb38dac13097c8103`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:38.874132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f3bc79c8-e9e3-4517-8b65-cd458081aea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:38.891944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:38.954730 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1397/2823] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:38 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:38.966395 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd31a49-f1bc-45d4-a119-8391b6e6569d admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:39.026418 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd31a49-f1bc-45d4-a119-8391b6e6569d admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:39.030139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d84d4-3993-481c-a4f8-0602825459cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:39.030762 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d84d4-3993-481c-a4f8-0602825459cb service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:39.031036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d84d4-3993-481c-a4f8-0602825459cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:39.031302 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d84d4-3993-481c-a4f8-0602825459cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:39.088276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d84d4-3993-481c-a4f8-0602825459cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:39.089774 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d84d4-3993-481c-a4f8-0602825459cb service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:39.099356 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1427/2824] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:40.806845 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc79c8-e9e3-4517-8b65-cd458081aea8 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:40.808402 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f3bc79c8-e9e3-4517-8b65-cd458081aea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:40.822171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc79c8-e9e3-4517-8b65-cd458081aea8 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:40.826993 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2b4cb-7913-46d3-9575-4071e021f971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:40.827611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2b4cb-7913-46d3-9575-4071e021f971 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:40.827894 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2b4cb-7913-46d3-9575-4071e021f971 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:40.828238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2b4cb-7913-46d3-9575-4071e021f971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:40.892855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2b4cb-7913-46d3-9575-4071e021f971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:40.894363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2b4cb-7913-46d3-9575-4071e021f971 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:40.903381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1398/2825] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:41.472571 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d84d4-3993-481c-a4f8-0602825459cb service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:41.481642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d84d4-3993-481c-a4f8-0602825459cb service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:41.485919 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957ac02-5f99-4dcb-a199-00a5926c6e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:41.486849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957ac02-5f99-4dcb-a199-00a5926c6e54 service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:41.487211 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957ac02-5f99-4dcb-a199-00a5926c6e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:41.487575 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957ac02-5f99-4dcb-a199-00a5926c6e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:41.498947 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957ac02-5f99-4dcb-a199-00a5926c6e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:41.500454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957ac02-5f99-4dcb-a199-00a5926c6e54 service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:41.510699 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1428/2826] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:41 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:48.633281 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2b4cb-7913-46d3-9575-4071e021f971 service cinder] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:48.699308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2b4cb-7913-46d3-9575-4071e021f971 service cinder] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:48.704266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:48.704901 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:48.705182 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:48.705450 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] PATH_INFO: `/v3/users/77a7ea623b2a463092fdb1ad73ae6ece` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:48.706697 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77a7ea623b2a463092fdb1ad73ae6ece)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:48.708607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:48.764822 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77a7ea623b2a463092fdb1ad73ae6ece'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:48.765616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77a7ea623b2a463092fdb1ad73ae6ece'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:48.771427 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:48.779931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77a7ea623b2a463092fdb1ad73ae6ece'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:48.786823 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77a7ea623b2a463092fdb1ad73ae6ece'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:48.795338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77a7ea623b2a463092fdb1ad73ae6ece'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:48.801586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1399/2827] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:48 2022] DELETE /identity/v3/users/77a7ea623b2a463092fdb1ad73ae6ece => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:48.806399 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4693d3-f5c3-4321-95c4-01c23af14a1e None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:48.806908 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4693d3-f5c3-4321-95c4-01c23af14a1e None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:48.807036 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4693d3-f5c3-4321-95c4-01c23af14a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:48.829022 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0c4693d3-f5c3-4321-95c4-01c23af14a1e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:48.834371 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0c4693d3-f5c3-4321-95c4-01c23af14a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:48.884152 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:48.898610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1429/2828] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:48.909941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:48.911396 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:48.979554 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f4678-2183-4afa-b2a8-3b5724d3f0a3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:48.983417 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54715f2-d834-41e9-a8d0-9be5373d9a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:48.984084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54715f2-d834-41e9-a8d0-9be5373d9a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:48.984382 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54715f2-d834-41e9-a8d0-9be5373d9a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:48.984672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54715f2-d834-41e9-a8d0-9be5373d9a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:48.994670 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54715f2-d834-41e9-a8d0-9be5373d9a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:48.996287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54715f2-d834-41e9-a8d0-9be5373d9a34 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:49.006547 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1400/2829] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:49.286388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4693d3-f5c3-4321-95c4-01c23af14a1e None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:49.287972 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0c4693d3-f5c3-4321-95c4-01c23af14a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:49.300926 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4693d3-f5c3-4321-95c4-01c23af14a1e None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:49.306809 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04024910-c078-483e-9566-724797d74da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:49.307596 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04024910-c078-483e-9566-724797d74da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:49.307856 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04024910-c078-483e-9566-724797d74da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:49.308129 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04024910-c078-483e-9566-724797d74da6 admin admin] PATH_INFO: `/v3/projects/91d9cbced64e4f73b37bc20c66579fc2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:49.309656 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04024910-c078-483e-9566-724797d74da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d9cbced64e4f73b37bc20c66579fc2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:49.311612 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04024910-c078-483e-9566-724797d74da6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:49.366476 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04024910-c078-483e-9566-724797d74da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d9cbced64e4f73b37bc20c66579fc2', 'request_id': 'req-04024910-c078-483e-9566-724797d74da6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.373421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04024910-c078-483e-9566-724797d74da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d9cbced64e4f73b37bc20c66579fc2', 'request_id': 'req-04024910-c078-483e-9566-724797d74da6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.379492 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04024910-c078-483e-9566-724797d74da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d9cbced64e4f73b37bc20c66579fc2', 'request_id': 'req-04024910-c078-483e-9566-724797d74da6'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.393883 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04024910-c078-483e-9566-724797d74da6 admin admin] The token cache is being invalidate because project 91d9cbced64e4f73b37bc20c66579fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:49.394167 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04024910-c078-483e-9566-724797d74da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.394833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-04024910-c078-483e-9566-724797d74da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.396360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1430/2830] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:49 2022] DELETE /identity/v3/projects/91d9cbced64e4f73b37bc20c66579fc2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:49.400494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54715f2-d834-41e9-a8d0-9be5373d9a34 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:49.410341 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54715f2-d834-41e9-a8d0-9be5373d9a34 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:49.467832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:49.468509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:49.468781 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:49.469060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] PATH_INFO: `/v3/users/07a53ad1569e4fa5bc3ce9e315f54acd` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:49.470260 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07a53ad1569e4fa5bc3ce9e315f54acd)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:49.472229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:49.521830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07a53ad1569e4fa5bc3ce9e315f54acd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.522486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07a53ad1569e4fa5bc3ce9e315f54acd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.534238 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07a53ad1569e4fa5bc3ce9e315f54acd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.541358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07a53ad1569e4fa5bc3ce9e315f54acd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.543932 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:49.549982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07a53ad1569e4fa5bc3ce9e315f54acd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.557346 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1401/2831] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:49 2022] DELETE /identity/v3/users/07a53ad1569e4fa5bc3ce9e315f54acd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:49.853871 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04024910-c078-483e-9566-724797d74da6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:49.855599 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-04024910-c078-483e-9566-724797d74da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:49.869983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04024910-c078-483e-9566-724797d74da6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:49.929829 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:49.930501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:49.930782 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:49.931050 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] PATH_INFO: `/v3/projects/be7ea3c46f44445b8def4edfdfd2ba17` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:49.932223 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be7ea3c46f44445b8def4edfdfd2ba17)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:49.934178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:49.980802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be7ea3c46f44445b8def4edfdfd2ba17', 'request_id': 'req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.986907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be7ea3c46f44445b8def4edfdfd2ba17', 'request_id': 'req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:49.992424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be7ea3c46f44445b8def4edfdfd2ba17', 'request_id': 'req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.009368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] The token cache is being invalidate because project be7ea3c46f44445b8def4edfdfd2ba17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:50.009368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.009368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.009368 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1431/2832] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:49 2022] DELETE /identity/v3/projects/be7ea3c46f44445b8def4edfdfd2ba17 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:50.013962 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:50.015665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:50.028858 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1601f2-4cdb-491d-aa55-7f900ebb7641 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:50.097583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:50.098304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.098589 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.098863 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] PATH_INFO: `/v3/users/bea1cc64dbdc4e0cb38dac13097c8103` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.100059 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bea1cc64dbdc4e0cb38dac13097c8103)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:50.105966 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:50.157945 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bea1cc64dbdc4e0cb38dac13097c8103'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.158644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bea1cc64dbdc4e0cb38dac13097c8103'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.172301 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bea1cc64dbdc4e0cb38dac13097c8103'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.179819 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bea1cc64dbdc4e0cb38dac13097c8103'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.189297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5caecc2b-f89e-413c-a1ad-0a1ad80ac15c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bea1cc64dbdc4e0cb38dac13097c8103'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.196672 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1402/2833] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:50 2022] DELETE /identity/v3/users/bea1cc64dbdc4e0cb38dac13097c8103 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:50.202321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:50.203819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:50.216703 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d9e79f-e3bf-4f57-8aab-90925fcb2c17 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:50.220475 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:50.284295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:50.284981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.285251 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.285520 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] PATH_INFO: `/v3/users/20046700233d4091bb343d391a1ac6da` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.286719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20046700233d4091bb343d391a1ac6da)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:50.288662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:50.342528 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20046700233d4091bb343d391a1ac6da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.343275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20046700233d4091bb343d391a1ac6da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.355559 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20046700233d4091bb343d391a1ac6da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.362540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20046700233d4091bb343d391a1ac6da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.376067 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20046700233d4091bb343d391a1ac6da'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.384071 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1432/2834] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:50 2022] DELETE /identity/v3/users/20046700233d4091bb343d391a1ac6da => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:50.389053 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59621264-7f4f-4c9c-a83f-ae01b4fd02a1 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.389443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59621264-7f4f-4c9c-a83f-ae01b4fd02a1 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.390000 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59621264-7f4f-4c9c-a83f-ae01b4fd02a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.415812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-59621264-7f4f-4c9c-a83f-ae01b4fd02a1 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:50.417037 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-59621264-7f4f-4c9c-a83f-ae01b4fd02a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:50.487099 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:50.488934 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:50.507259 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1403/2835] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:50.514209 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59621264-7f4f-4c9c-a83f-ae01b4fd02a1 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:50.515692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-59621264-7f4f-4c9c-a83f-ae01b4fd02a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:50.530943 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:50.573946 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b240a6c-8aeb-4bbe-ada5-60ed9b3dd5e7 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:50.578566 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:50.579217 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.579494 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.579764 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] PATH_INFO: `/v3/projects/e3d3b7245bc7468c9fd0f7ee8f6fe34a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.580948 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d3b7245bc7468c9fd0f7ee8f6fe34a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:50.583057 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:50.584709 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59621264-7f4f-4c9c-a83f-ae01b4fd02a1 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:50.588513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d23938-4d4f-4822-8052-a646750cb07c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:50.589135 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d23938-4d4f-4822-8052-a646750cb07c service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.589416 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d23938-4d4f-4822-8052-a646750cb07c service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.589689 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d23938-4d4f-4822-8052-a646750cb07c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.599120 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d23938-4d4f-4822-8052-a646750cb07c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:50.600530 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d23938-4d4f-4822-8052-a646750cb07c service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:50.610335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1404/2836] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:50.616989 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:50.636530 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d3b7245bc7468c9fd0f7ee8f6fe34a', 'request_id': 'req-6baceea7-5e23-48a4-8402-5e2ba5912139'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.642940 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d3b7245bc7468c9fd0f7ee8f6fe34a', 'request_id': 'req-6baceea7-5e23-48a4-8402-5e2ba5912139'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.648802 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d3b7245bc7468c9fd0f7ee8f6fe34a', 'request_id': 'req-6baceea7-5e23-48a4-8402-5e2ba5912139'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.661336 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] The token cache is being invalidate because project e3d3b7245bc7468c9fd0f7ee8f6fe34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:50.661549 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.662221 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6baceea7-5e23-48a4-8402-5e2ba5912139 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:50.663851 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1433/2837] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:50 2022] DELETE /identity/v3/projects/e3d3b7245bc7468c9fd0f7ee8f6fe34a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:50.668988 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0be776-4515-4ab8-8c27-30cea4653b1e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.669276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0be776-4515-4ab8-8c27-30cea4653b1e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.669550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0be776-4515-4ab8-8c27-30cea4653b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.693747 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0d0be776-4515-4ab8-8c27-30cea4653b1e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:50.695267 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0d0be776-4515-4ab8-8c27-30cea4653b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:50.767760 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1405/2838] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:50.771605 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cbdcb-35de-4cc5-88e5-b8c9a31861a0 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.772026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cbdcb-35de-4cc5-88e5-b8c9a31861a0 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.772372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719cbdcb-35de-4cc5-88e5-b8c9a31861a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.797043 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-719cbdcb-35de-4cc5-88e5-b8c9a31861a0 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:50.798428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-719cbdcb-35de-4cc5-88e5-b8c9a31861a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:50.806916 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1434/2839] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:50.811499 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0be776-4515-4ab8-8c27-30cea4653b1e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:50.822403 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0be776-4515-4ab8-8c27-30cea4653b1e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:50.827247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d647cf-66c3-41ae-a393-a723489fd6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:50.827903 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d647cf-66c3-41ae-a393-a723489fd6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.828276 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d647cf-66c3-41ae-a393-a723489fd6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.828643 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d647cf-66c3-41ae-a393-a723489fd6d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.830104 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d647cf-66c3-41ae-a393-a723489fd6d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:01:50.830415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d647cf-66c3-41ae-a393-a723489fd6d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:50.832148 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d647cf-66c3-41ae-a393-a723489fd6d0 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:50.845106 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1406/2840] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:01:50.849888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719cbdcb-35de-4cc5-88e5-b8c9a31861a0 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:50.860011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719cbdcb-35de-4cc5-88e5-b8c9a31861a0 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:50.864597 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740218ba-b9d5-49be-b4a1-32d7bbd67ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:50.865216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740218ba-b9d5-49be-b4a1-32d7bbd67ff9 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.865500 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740218ba-b9d5-49be-b4a1-32d7bbd67ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.865773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740218ba-b9d5-49be-b4a1-32d7bbd67ff9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.866515 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740218ba-b9d5-49be-b4a1-32d7bbd67ff9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:50.868425 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740218ba-b9d5-49be-b4a1-32d7bbd67ff9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:50.902117 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1435/2841] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:50 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:50.903711 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740218ba-b9d5-49be-b4a1-32d7bbd67ff9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:50.903878 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d647cf-66c3-41ae-a393-a723489fd6d0 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:50.905507 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-740218ba-b9d5-49be-b4a1-32d7bbd67ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:50.919954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d647cf-66c3-41ae-a393-a723489fd6d0 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:50.980861 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740218ba-b9d5-49be-b4a1-32d7bbd67ff9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:50.981500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5459c3-8780-41ca-9d7c-70f770c662d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:50.982167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5459c3-8780-41ca-9d7c-70f770c662d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.982451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5459c3-8780-41ca-9d7c-70f770c662d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.982729 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5459c3-8780-41ca-9d7c-70f770c662d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.983405 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5459c3-8780-41ca-9d7c-70f770c662d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:50.985327 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5459c3-8780-41ca-9d7c-70f770c662d8 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:50.985450 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:50.986174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:50.986453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:50.986725 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] PATH_INFO: `/v3/projects/fee28ced1ed34219b4829d1ff3ca3f20` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:50.987907 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee28ced1ed34219b4829d1ff3ca3f20)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:50.989812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:51.007550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1407/2842] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:50 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:51.011522 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5459c3-8780-41ca-9d7c-70f770c662d8 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:51.021466 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5459c3-8780-41ca-9d7c-70f770c662d8 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:51.026965 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd2bfc-0a1d-4478-9fbe-e939d337c2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:51.027566 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd2bfc-0a1d-4478-9fbe-e939d337c2c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.027852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd2bfc-0a1d-4478-9fbe-e939d337c2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.028124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd2bfc-0a1d-4478-9fbe-e939d337c2c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.028817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd2bfc-0a1d-4478-9fbe-e939d337c2c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:51.030526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd2bfc-0a1d-4478-9fbe-e939d337c2c5 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:51.037678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee28ced1ed34219b4829d1ff3ca3f20', 'request_id': 'req-9ab45432-c391-497e-b9aa-3540b92c247b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.041153 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1408/2843] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:51.044213 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee28ced1ed34219b4829d1ff3ca3f20', 'request_id': 'req-9ab45432-c391-497e-b9aa-3540b92c247b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.045414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd2bfc-0a1d-4478-9fbe-e939d337c2c5 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:51.056212 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee28ced1ed34219b4829d1ff3ca3f20', 'request_id': 'req-9ab45432-c391-497e-b9aa-3540b92c247b'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.057131 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd2bfc-0a1d-4478-9fbe-e939d337c2c5 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:51.069150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] The token cache is being invalidate because project fee28ced1ed34219b4829d1ff3ca3f20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:51.069675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.070224 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.072155 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1436/2844] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:50 2022] DELETE /identity/v3/projects/fee28ced1ed34219b4829d1ff3ca3f20 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:51.075914 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:51.076579 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:51.078442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:51.098041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:51.114644 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3a3b1-dbdf-4e49-84b0-c945fb8d68ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:51.115264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a3b1-dbdf-4e49-84b0-c945fb8d68ad admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.115542 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a3b1-dbdf-4e49-84b0-c945fb8d68ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.115815 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3a3b1-dbdf-4e49-84b0-c945fb8d68ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.116510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3a3b1-dbdf-4e49-84b0-c945fb8d68ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:51.118171 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3a3b1-dbdf-4e49-84b0-c945fb8d68ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:51.128254 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1409/2845] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:51.132305 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3a3b1-dbdf-4e49-84b0-c945fb8d68ad admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:51.134855 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-55e3a3b1-dbdf-4e49-84b0-c945fb8d68ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:51.149378 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3a3b1-dbdf-4e49-84b0-c945fb8d68ad admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:51.152158 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab45432-c391-497e-b9aa-3540b92c247b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:51.154112 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee12afe-ee7c-49cb-beeb-9c8797960b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:51.154792 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee12afe-ee7c-49cb-beeb-9c8797960b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.155695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee12afe-ee7c-49cb-beeb-9c8797960b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.155695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee12afe-ee7c-49cb-beeb-9c8797960b46 admin admin] PATH_INFO: `/v3/projects/11c3b80263c94926948bb32dbca03c37/users/c0a63370cf974a60afc8d31f6561214f/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.158303 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:51.158961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.159263 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.159584 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] PATH_INFO: `/v3/users/3034bfa2aef6400f82127799f0e6bdcb` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.160981 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=3034bfa2aef6400f82127799f0e6bdcb)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:51.162888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:51.171381 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee12afe-ee7c-49cb-beeb-9c8797960b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c3b80263c94926948bb32dbca03c37, user_id=c0a63370cf974a60afc8d31f6561214f, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:51.174056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee12afe-ee7c-49cb-beeb-9c8797960b46 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:51.189595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1410/2846] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:51 2022] PUT /identity/v3/projects/11c3b80263c94926948bb32dbca03c37/users/c0a63370cf974a60afc8d31f6561214f/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:51.200756 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72b291-4544-4c23-bdae-e96bc574106f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.201045 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72b291-4544-4c23-bdae-e96bc574106f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.201319 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d72b291-4544-4c23-bdae-e96bc574106f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.221006 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3034bfa2aef6400f82127799f0e6bdcb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.221678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3034bfa2aef6400f82127799f0e6bdcb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.229803 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5d72b291-4544-4c23-bdae-e96bc574106f None None] MFA Rules not processed for user `c0a63370cf974a60afc8d31f6561214f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:51.231012 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d72b291-4544-4c23-bdae-e96bc574106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:51.233834 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3034bfa2aef6400f82127799f0e6bdcb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.240618 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3034bfa2aef6400f82127799f0e6bdcb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.248831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3034bfa2aef6400f82127799f0e6bdcb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.255564 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1437/2847] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:51 2022] DELETE /identity/v3/users/3034bfa2aef6400f82127799f0e6bdcb => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:51.298896 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1411/2848] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:51.310740 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:51.312268 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:51.374478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198ec506-c277-4792-8ef6-2d693b8b03ee admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:51.378746 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f940627-f4a4-4d9a-b893-824da81932aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:51.379249 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f940627-f4a4-4d9a-b893-824da81932aa service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.379527 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f940627-f4a4-4d9a-b893-824da81932aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.379814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f940627-f4a4-4d9a-b893-824da81932aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.432776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:51.446222 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f940627-f4a4-4d9a-b893-824da81932aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:51.448126 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f940627-f4a4-4d9a-b893-824da81932aa service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:51.462189 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1438/2849] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:51.556606 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72b291-4544-4c23-bdae-e96bc574106f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:51.558105 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d72b291-4544-4c23-bdae-e96bc574106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:51.634138 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d72b291-4544-4c23-bdae-e96bc574106f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:51.662665 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:51.693217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:51.694003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.694322 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.694629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] PATH_INFO: `/v3/projects/fd0386f1081147bdb93cbd78ac8dab64` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.696432 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0386f1081147bdb93cbd78ac8dab64)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:51.698731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:51.756549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0386f1081147bdb93cbd78ac8dab64', 'request_id': 'req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.763384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0386f1081147bdb93cbd78ac8dab64', 'request_id': 'req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.771422 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0386f1081147bdb93cbd78ac8dab64', 'request_id': 'req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.786180 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] The token cache is being invalidate because project fd0386f1081147bdb93cbd78ac8dab64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:51.786514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.787266 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-5ae88e3f-d984-46fc-bdc0-d9294ebe9db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:51.793051 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1412/2850] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:51 2022] DELETE /identity/v3/projects/fd0386f1081147bdb93cbd78ac8dab64 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:51.794660 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905f34d-7c7d-4cd8-99e6-775d8dcaedae None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.794997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905f34d-7c7d-4cd8-99e6-775d8dcaedae None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.795229 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905f34d-7c7d-4cd8-99e6-775d8dcaedae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.819028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-c905f34d-7c7d-4cd8-99e6-775d8dcaedae None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:51.820271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-c905f34d-7c7d-4cd8-99e6-775d8dcaedae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:51.887833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1439/2851] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:51.895782 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d59f43-e4dd-44f8-b0f0-05994b3d0bae None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.896073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d59f43-e4dd-44f8-b0f0-05994b3d0bae None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.896364 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d59f43-e4dd-44f8-b0f0-05994b3d0bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.916086 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-15d59f43-e4dd-44f8-b0f0-05994b3d0bae None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:51.917290 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-15d59f43-e4dd-44f8-b0f0-05994b3d0bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:51.925306 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1413/2852] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:51.929675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905f34d-7c7d-4cd8-99e6-775d8dcaedae None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:51.939979 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905f34d-7c7d-4cd8-99e6-775d8dcaedae None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:51.944736 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e08c0-eb59-4e6a-9ab1-61594f853a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:51.945726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e08c0-eb59-4e6a-9ab1-61594f853a06 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.946030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e08c0-eb59-4e6a-9ab1-61594f853a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.946300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e08c0-eb59-4e6a-9ab1-61594f853a06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.947105 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e08c0-eb59-4e6a-9ab1-61594f853a06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:01:51.947408 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e08c0-eb59-4e6a-9ab1-61594f853a06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:51.948990 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95e08c0-eb59-4e6a-9ab1-61594f853a06 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:51.960495 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1440/2853] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:01:51.964812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d59f43-e4dd-44f8-b0f0-05994b3d0bae None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:51.974228 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d59f43-e4dd-44f8-b0f0-05994b3d0bae None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:51.979061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bf41d-79ad-40e4-b059-ace2aebda7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:51.979673 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bf41d-79ad-40e4-b059-ace2aebda7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:51.979944 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bf41d-79ad-40e4-b059-ace2aebda7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:51.980218 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bf41d-79ad-40e4-b059-ace2aebda7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:51.980884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bf41d-79ad-40e4-b059-ace2aebda7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:51.982824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bf41d-79ad-40e4-b059-ace2aebda7a3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.010936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1414/2854] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:51 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:52.015030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e08c0-eb59-4e6a-9ab1-61594f853a06 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.025253 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e08c0-eb59-4e6a-9ab1-61594f853a06 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.081906 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376d8c3-bcce-43c1-abb8-d4ca5310e28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.082598 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376d8c3-bcce-43c1-abb8-d4ca5310e28a admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.082913 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376d8c3-bcce-43c1-abb8-d4ca5310e28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.083308 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9376d8c3-bcce-43c1-abb8-d4ca5310e28a admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.084150 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376d8c3-bcce-43c1-abb8-d4ca5310e28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:52.086200 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9376d8c3-bcce-43c1-abb8-d4ca5310e28a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.107828 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1441/2855] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:52 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:52.112124 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bf41d-79ad-40e4-b059-ace2aebda7a3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.122883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bf41d-79ad-40e4-b059-ace2aebda7a3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.128198 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fc9a2-c773-4039-b79a-23bf9589a969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.128884 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fc9a2-c773-4039-b79a-23bf9589a969 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.129279 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fc9a2-c773-4039-b79a-23bf9589a969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.129657 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fc9a2-c773-4039-b79a-23bf9589a969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.130456 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fc9a2-c773-4039-b79a-23bf9589a969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:52.132123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005fc9a2-c773-4039-b79a-23bf9589a969 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.142362 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1415/2856] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:01:52.146349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376d8c3-bcce-43c1-abb8-d4ca5310e28a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.156961 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9376d8c3-bcce-43c1-abb8-d4ca5310e28a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.163514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb2599-3b0f-4d12-806f-32c1406a6e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.164142 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb2599-3b0f-4d12-806f-32c1406a6e34 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.164424 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb2599-3b0f-4d12-806f-32c1406a6e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.164697 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb2599-3b0f-4d12-806f-32c1406a6e34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.165396 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb2599-3b0f-4d12-806f-32c1406a6e34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:52.167216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fb2599-3b0f-4d12-806f-32c1406a6e34 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.178638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1442/2857] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:52.182772 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fc9a2-c773-4039-b79a-23bf9589a969 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.194363 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fc9a2-c773-4039-b79a-23bf9589a969 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.199514 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1812097-4adb-4399-bfc8-4d51f1a7cc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.200179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1812097-4adb-4399-bfc8-4d51f1a7cc5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.200474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1812097-4adb-4399-bfc8-4d51f1a7cc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.200731 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1812097-4adb-4399-bfc8-4d51f1a7cc5d admin admin] PATH_INFO: `/v3/projects/a6847b5304354552b7efb0405700b3d6/users/f6110999187a4c0fa046dbae4e83b7b2/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.215635 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1812097-4adb-4399-bfc8-4d51f1a7cc5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6847b5304354552b7efb0405700b3d6, user_id=f6110999187a4c0fa046dbae4e83b7b2, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:52.218478 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1812097-4adb-4399-bfc8-4d51f1a7cc5d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.232315 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1416/2858] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:52 2022] PUT /identity/v3/projects/a6847b5304354552b7efb0405700b3d6/users/f6110999187a4c0fa046dbae4e83b7b2/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:52.243798 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf59eda-5729-4b72-a6bb-b997169d7221 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.244086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf59eda-5729-4b72-a6bb-b997169d7221 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.244360 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf59eda-5729-4b72-a6bb-b997169d7221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.271243 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-ddf59eda-5729-4b72-a6bb-b997169d7221 None None] MFA Rules not processed for user `f6110999187a4c0fa046dbae4e83b7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:52.272478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ddf59eda-5729-4b72-a6bb-b997169d7221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:52.316525 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:52.338701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1443/2859] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:52.343250 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1812097-4adb-4399-bfc8-4d51f1a7cc5d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.401240 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:52.404883 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1812097-4adb-4399-bfc8-4d51f1a7cc5d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.411790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d120d-dce2-4b71-b503-49cc5c1dc377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.412832 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d120d-dce2-4b71-b503-49cc5c1dc377 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.413129 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d120d-dce2-4b71-b503-49cc5c1dc377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.413647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335d120d-dce2-4b71-b503-49cc5c1dc377 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.414546 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d120d-dce2-4b71-b503-49cc5c1dc377 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:52.416470 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335d120d-dce2-4b71-b503-49cc5c1dc377 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.446726 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1417/2860] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:01:52 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:01:52.451153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf59eda-5729-4b72-a6bb-b997169d7221 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.462351 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf59eda-5729-4b72-a6bb-b997169d7221 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.522773 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed168699-23eb-48cd-92f1-68b869f7e4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.523420 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed168699-23eb-48cd-92f1-68b869f7e4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.523706 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed168699-23eb-48cd-92f1-68b869f7e4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.523983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed168699-23eb-48cd-92f1-68b869f7e4a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.524650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed168699-23eb-48cd-92f1-68b869f7e4a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:52.526602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed168699-23eb-48cd-92f1-68b869f7e4a1 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.548670 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1444/2861] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:01:52 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:01:52.552764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d120d-dce2-4b71-b503-49cc5c1dc377 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.563358 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335d120d-dce2-4b71-b503-49cc5c1dc377 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.567971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83099299-031e-4d13-9dbb-010fba48fc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.568583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83099299-031e-4d13-9dbb-010fba48fc1d admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.568867 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83099299-031e-4d13-9dbb-010fba48fc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.569144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83099299-031e-4d13-9dbb-010fba48fc1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.569881 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83099299-031e-4d13-9dbb-010fba48fc1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:52.571847 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83099299-031e-4d13-9dbb-010fba48fc1d admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.582330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1418/2862] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:52.586343 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed168699-23eb-48cd-92f1-68b869f7e4a1 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.596209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed168699-23eb-48cd-92f1-68b869f7e4a1 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.601179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbd96c-12e2-4368-8acf-36b7f502e3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.601860 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbd96c-12e2-4368-8acf-36b7f502e3ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.602139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbd96c-12e2-4368-8acf-36b7f502e3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.602411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbd96c-12e2-4368-8acf-36b7f502e3ba admin admin] PATH_INFO: `/v3/projects/057ba20dc2ad4659a7d236e70315d01a/users/dd81260255654c34a3ff777bee15194a/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.616712 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbd96c-12e2-4368-8acf-36b7f502e3ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=057ba20dc2ad4659a7d236e70315d01a, user_id=dd81260255654c34a3ff777bee15194a, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:52.619499 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbd96c-12e2-4368-8acf-36b7f502e3ba admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.634696 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1445/2863] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:52 2022] PUT /identity/v3/projects/057ba20dc2ad4659a7d236e70315d01a/users/dd81260255654c34a3ff777bee15194a/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:52.638616 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83099299-031e-4d13-9dbb-010fba48fc1d admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.648611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83099299-031e-4d13-9dbb-010fba48fc1d admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.704866 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5340943d-b996-4e9d-bf8c-a066e10e2495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.705411 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340943d-b996-4e9d-bf8c-a066e10e2495 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.705692 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340943d-b996-4e9d-bf8c-a066e10e2495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.706002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5340943d-b996-4e9d-bf8c-a066e10e2495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.706746 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340943d-b996-4e9d-bf8c-a066e10e2495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:52.708338 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5340943d-b996-4e9d-bf8c-a066e10e2495 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.718041 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1419/2864] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:01:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:01:52.722433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbd96c-12e2-4368-8acf-36b7f502e3ba admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.732534 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbd96c-12e2-4368-8acf-36b7f502e3ba admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.737548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a5f7e-2e59-4780-bd48-374d7a670697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.738272 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a5f7e-2e59-4780-bd48-374d7a670697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.738540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a5f7e-2e59-4780-bd48-374d7a670697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.738815 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a5f7e-2e59-4780-bd48-374d7a670697 admin admin] PATH_INFO: `/v3/projects/057ba20dc2ad4659a7d236e70315d01a/users/dd81260255654c34a3ff777bee15194a/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.740817 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a5f7e-2e59-4780-bd48-374d7a670697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=057ba20dc2ad4659a7d236e70315d01a, user_id=dd81260255654c34a3ff777bee15194a, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:52.743493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97a5f7e-2e59-4780-bd48-374d7a670697 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:52.754195 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1446/2865] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:01:52 2022] PUT /identity/v3/projects/057ba20dc2ad4659a7d236e70315d01a/users/dd81260255654c34a3ff777bee15194a/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:01:52.765165 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639af165-6b61-49ea-8bee-9dc8d5cd159f None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.765438 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639af165-6b61-49ea-8bee-9dc8d5cd159f None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.765717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639af165-6b61-49ea-8bee-9dc8d5cd159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:52.793513 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-639af165-6b61-49ea-8bee-9dc8d5cd159f None None] MFA Rules not processed for user `dd81260255654c34a3ff777bee15194a`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:52.794764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-639af165-6b61-49ea-8bee-9dc8d5cd159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:52.864695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1420/2866] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:52 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:01:52.876035 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a5f7e-2e59-4780-bd48-374d7a670697 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:52.878185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-e97a5f7e-2e59-4780-bd48-374d7a670697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:52.949174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a5f7e-2e59-4780-bd48-374d7a670697 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:52.953100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da324e53-c81e-4910-919d-c0d55e5e735c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:52.953755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324e53-c81e-4910-919d-c0d55e5e735c service cinder] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:52.954042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324e53-c81e-4910-919d-c0d55e5e735c service cinder] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:52.954315 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324e53-c81e-4910-919d-c0d55e5e735c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:53.007372 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:53.011170 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da324e53-c81e-4910-919d-c0d55e5e735c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:53.012923 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da324e53-c81e-4910-919d-c0d55e5e735c service cinder] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:53.024271 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1447/2867] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:57.303957 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639af165-6b61-49ea-8bee-9dc8d5cd159f None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:57.305635 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-639af165-6b61-49ea-8bee-9dc8d5cd159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:57.318624 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639af165-6b61-49ea-8bee-9dc8d5cd159f None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:57.322999 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4454789-d8c0-47d8-bef0-081e7ddcf5a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'becff3f1dacb43c8a014c95456e86bc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:57.323685 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4454789-d8c0-47d8-bef0-081e7ddcf5a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:57.324556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4454789-d8c0-47d8-bef0-081e7ddcf5a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:57.324556 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4454789-d8c0-47d8-bef0-081e7ddcf5a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:57.337130 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4454789-d8c0-47d8-bef0-081e7ddcf5a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:57.338597 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4454789-d8c0-47d8-bef0-081e7ddcf5a8 service cinder] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:57.348933 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1421/2868] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:01:58.148412 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da324e53-c81e-4910-919d-c0d55e5e735c service cinder] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:58.150147 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-da324e53-c81e-4910-919d-c0d55e5e735c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:58.218391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da324e53-c81e-4910-919d-c0d55e5e735c service cinder] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:58.223011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:58.223760 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:58.224047 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:58.224322 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] PATH_INFO: `/v3/users/23c72c39761642f9982efefe6e074906` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:58.225512 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c72c39761642f9982efefe6e074906)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:58.227483 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:58.277088 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c72c39761642f9982efefe6e074906'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:58.277761 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c72c39761642f9982efefe6e074906'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:58.289421 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c72c39761642f9982efefe6e074906'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:58.296209 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c72c39761642f9982efefe6e074906'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:58.304973 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c72c39761642f9982efefe6e074906'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:58.311845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1448/2869] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:01:58 2022] DELETE /identity/v3/users/23c72c39761642f9982efefe6e074906 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:01:58.316895 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e20dc-3337-4c7b-8eed-da1b6362586a None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:58.317469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e20dc-3337-4c7b-8eed-da1b6362586a None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:58.317565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e20dc-3337-4c7b-8eed-da1b6362586a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:58.338065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-6d2e20dc-3337-4c7b-8eed-da1b6362586a None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:01:58.339296 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d2e20dc-3337-4c7b-8eed-da1b6362586a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:58.377777 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:01:58.407291 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1422/2870] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:01:58.418849 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:58.420702 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:58.491238 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f28186f-8e7b-449f-b7ce-d897e37a3e5a admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:58.495684 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c81de0-d41a-4c88-9538-18fa18f4c554 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:58.496315 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c81de0-d41a-4c88-9538-18fa18f4c554 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:58.496625 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c81de0-d41a-4c88-9538-18fa18f4c554 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:58.496898 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c81de0-d41a-4c88-9538-18fa18f4c554 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:58.506610 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c81de0-d41a-4c88-9538-18fa18f4c554 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:58.508049 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c81de0-d41a-4c88-9538-18fa18f4c554 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:58.518015 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1449/2871] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:01:58.799550 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e20dc-3337-4c7b-8eed-da1b6362586a None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:01:58.813776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d2e20dc-3337-4c7b-8eed-da1b6362586a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:01:58.814579 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e20dc-3337-4c7b-8eed-da1b6362586a None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:01:58.819833 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:01:58.820874 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:01:58.821217 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:01:58.821527 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] PATH_INFO: `/v3/projects/51d5adeabc614cf48df9369517491baf` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:01:58.822794 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51d5adeabc614cf48df9369517491baf)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:01:58.824715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:01:58.883749 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51d5adeabc614cf48df9369517491baf', 'request_id': 'req-ffbf183d-ce33-46c2-af26-df6d04afb931'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:58.889440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51d5adeabc614cf48df9369517491baf', 'request_id': 'req-ffbf183d-ce33-46c2-af26-df6d04afb931'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:58.898844 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51d5adeabc614cf48df9369517491baf', 'request_id': 'req-ffbf183d-ce33-46c2-af26-df6d04afb931'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:58.913415 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] The token cache is being invalidate because project 51d5adeabc614cf48df9369517491baf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:01:58.913703 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:58.914481 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-ffbf183d-ce33-46c2-af26-df6d04afb931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:01:58.916637 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1423/2872] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:01:58 2022] DELETE /identity/v3/projects/51d5adeabc614cf48df9369517491baf => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:02:01.505545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c81de0-d41a-4c88-9538-18fa18f4c554 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:01.507233 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05c81de0-d41a-4c88-9538-18fa18f4c554 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:01.520558 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c81de0-d41a-4c88-9538-18fa18f4c554 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:01.533389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:02:01.581098 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:01.581765 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:01.582063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:01.582334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] PATH_INFO: `/v3/users/c0a63370cf974a60afc8d31f6561214f` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:01.583624 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0a63370cf974a60afc8d31f6561214f)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:01.585606 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:01.634679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0a63370cf974a60afc8d31f6561214f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:01.635334 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0a63370cf974a60afc8d31f6561214f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:01.648506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0a63370cf974a60afc8d31f6561214f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:01.656111 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0a63370cf974a60afc8d31f6561214f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:01.664688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0a63370cf974a60afc8d31f6561214f'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:01.670889 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1450/2873] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:02:01 2022] DELETE /identity/v3/users/c0a63370cf974a60afc8d31f6561214f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:02:01.675631 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1190b3c-ef8f-4bc1-bfad-c4b4eabd3a06 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:01.676179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1190b3c-ef8f-4bc1-bfad-c4b4eabd3a06 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:01.676179 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1190b3c-ef8f-4bc1-bfad-c4b4eabd3a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:01.695789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c1190b3c-ef8f-4bc1-bfad-c4b4eabd3a06 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:02:01.696882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1190b3c-ef8f-4bc1-bfad-c4b4eabd3a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:01.757989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1424/2874] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:02:01.768020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:01.769562 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:01.829355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b69943d-77fe-479f-bcfc-c47c8da605d9 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:01.833465 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d58818-6a14-488f-b7ec-0092db0052c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:01.834135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d58818-6a14-488f-b7ec-0092db0052c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:01.834388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d58818-6a14-488f-b7ec-0092db0052c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:01.834657 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d58818-6a14-488f-b7ec-0092db0052c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:01.843943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d58818-6a14-488f-b7ec-0092db0052c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:01.845374 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d58818-6a14-488f-b7ec-0092db0052c9 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:01.854915 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1451/2875] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:02:02.140158 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1190b3c-ef8f-4bc1-bfad-c4b4eabd3a06 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:02.141880 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c1190b3c-ef8f-4bc1-bfad-c4b4eabd3a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:02.154721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1190b3c-ef8f-4bc1-bfad-c4b4eabd3a06 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:02.160145 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:02.160793 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:02.161071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:02.161345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] PATH_INFO: `/v3/projects/11c3b80263c94926948bb32dbca03c37` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:02.162628 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c3b80263c94926948bb32dbca03c37)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:02.164613 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:02.212567 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c3b80263c94926948bb32dbca03c37', 'request_id': 'req-090ab7db-19d3-4a97-84e3-93e53cffb4dd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:02.218031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c3b80263c94926948bb32dbca03c37', 'request_id': 'req-090ab7db-19d3-4a97-84e3-93e53cffb4dd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:02.219640 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:02:02.227304 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c3b80263c94926948bb32dbca03c37', 'request_id': 'req-090ab7db-19d3-4a97-84e3-93e53cffb4dd'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:02.239328 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] The token cache is being invalidate because project 11c3b80263c94926948bb32dbca03c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:02:02.239591 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:02.240289 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-090ab7db-19d3-4a97-84e3-93e53cffb4dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:02.242570 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1425/2876] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:02:02 2022] DELETE /identity/v3/projects/11c3b80263c94926948bb32dbca03c37 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:02:02.320772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d58818-6a14-488f-b7ec-0092db0052c9 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:02.322467 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-35d58818-6a14-488f-b7ec-0092db0052c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:02.338304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d58818-6a14-488f-b7ec-0092db0052c9 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:02.402778 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:02:02.443068 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:02.446732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:02.446732 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:02.447031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] PATH_INFO: `/v3/users/f6110999187a4c0fa046dbae4e83b7b2` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:02.448552 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6110999187a4c0fa046dbae4e83b7b2)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:02.450759 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:02.531621 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6110999187a4c0fa046dbae4e83b7b2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:02.532365 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6110999187a4c0fa046dbae4e83b7b2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:02.545963 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6110999187a4c0fa046dbae4e83b7b2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:02.553872 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6110999187a4c0fa046dbae4e83b7b2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:02.567731 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6110999187a4c0fa046dbae4e83b7b2'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:02.577502 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1452/2877] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:02:02 2022] DELETE /identity/v3/users/f6110999187a4c0fa046dbae4e83b7b2 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:02:02.582184 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0e92b-e64f-471a-a956-de3c67ae3249 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:02.582474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0e92b-e64f-471a-a956-de3c67ae3249 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:02.582751 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0e92b-e64f-471a-a956-de3c67ae3249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:02.602603 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-5bc0e92b-e64f-471a-a956-de3c67ae3249 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:02:02.603812 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bc0e92b-e64f-471a-a956-de3c67ae3249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:02.663192 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1426/2878] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:02:02.674651 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:02.676153 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:02.736949 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e67d22-b0fa-42a6-8cff-777818401dcd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:02.741323 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510297a8-07c9-4e19-9d63-cb3a5a62359b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:02.742011 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510297a8-07c9-4e19-9d63-cb3a5a62359b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:02.742274 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510297a8-07c9-4e19-9d63-cb3a5a62359b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:02.742545 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510297a8-07c9-4e19-9d63-cb3a5a62359b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:02.752478 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510297a8-07c9-4e19-9d63-cb3a5a62359b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:02.754017 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510297a8-07c9-4e19-9d63-cb3a5a62359b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:02.765237 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1453/2879] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:02:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:02:03.461235 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0e92b-e64f-471a-a956-de3c67ae3249 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:03.462888 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5bc0e92b-e64f-471a-a956-de3c67ae3249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:03.478811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0e92b-e64f-471a-a956-de3c67ae3249 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:03.484084 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:03.484754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:03.485036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:03.485307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] PATH_INFO: `/v3/projects/a6847b5304354552b7efb0405700b3d6` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:03.486789 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6847b5304354552b7efb0405700b3d6)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:03.488846 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:03.546795 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6847b5304354552b7efb0405700b3d6', 'request_id': 'req-7e9621fe-1145-41b7-a397-507e527bd8ab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:03.553076 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6847b5304354552b7efb0405700b3d6', 'request_id': 'req-7e9621fe-1145-41b7-a397-507e527bd8ab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:03.564098 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6847b5304354552b7efb0405700b3d6', 'request_id': 'req-7e9621fe-1145-41b7-a397-507e527bd8ab'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:03.575931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] The token cache is being invalidate because project a6847b5304354552b7efb0405700b3d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:02:03.576214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:03.576852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:03.578400 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1427/2880] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:02:03 2022] DELETE /identity/v3/projects/a6847b5304354552b7efb0405700b3d6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:02:03.582557 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510297a8-07c9-4e19-9d63-cb3a5a62359b service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:03.592577 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510297a8-07c9-4e19-9d63-cb3a5a62359b service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:03.649784 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:03.650466 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:03.650719 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:03.651020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] PATH_INFO: `/v3/users/dd81260255654c34a3ff777bee15194a` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:03.652180 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd81260255654c34a3ff777bee15194a)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:03.654140 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:03.655698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:02:03.704300 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd81260255654c34a3ff777bee15194a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:03.704942 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd81260255654c34a3ff777bee15194a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:03.720340 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd81260255654c34a3ff777bee15194a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:03.727506 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd81260255654c34a3ff777bee15194a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:03.736401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-b94064d4-3b3d-4bfd-bdbb-6c71a3e183e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd81260255654c34a3ff777bee15194a'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:03.742839 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1454/2881] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:02:03 2022] DELETE /identity/v3/users/dd81260255654c34a3ff777bee15194a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:02:04.022132 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:04.023666 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:04.038630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9621fe-1145-41b7-a397-507e527bd8ab admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:04.096278 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:04.096946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.097226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.097500 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] PATH_INFO: `/v3/projects/057ba20dc2ad4659a7d236e70315d01a` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.098831 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=057ba20dc2ad4659a7d236e70315d01a)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:04.100830 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:04.112768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:02:04.157031 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '057ba20dc2ad4659a7d236e70315d01a', 'request_id': 'req-6492922b-af28-4753-8da2-2728c6258073'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:04.165539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '057ba20dc2ad4659a7d236e70315d01a', 'request_id': 'req-6492922b-af28-4753-8da2-2728c6258073'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:04.174836 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '057ba20dc2ad4659a7d236e70315d01a', 'request_id': 'req-6492922b-af28-4753-8da2-2728c6258073'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:04.186167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] The token cache is being invalidate because project 057ba20dc2ad4659a7d236e70315d01a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:02:04.186563 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:04.187214 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-6492922b-af28-4753-8da2-2728c6258073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:04.188687 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1428/2882] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:02:04 2022] DELETE /identity/v3/projects/057ba20dc2ad4659a7d236e70315d01a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:02:04.196409 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678af92-0b20-4d56-b45e-07c3e8a9b596 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.196701 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678af92-0b20-4d56-b45e-07c3e8a9b596 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.197008 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678af92-0b20-4d56-b45e-07c3e8a9b596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.217391 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-0678af92-0b20-4d56-b45e-07c3e8a9b596 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:02:04.218617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-0678af92-0b20-4d56-b45e-07c3e8a9b596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:04.277864 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1455/2883] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:02:04.287404 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bfb41-6e54-4117-9a7f-25d0cbc6e66d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.287696 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bfb41-6e54-4117-9a7f-25d0cbc6e66d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.288060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bfb41-6e54-4117-9a7f-25d0cbc6e66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.308220 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-0c2bfb41-6e54-4117-9a7f-25d0cbc6e66d None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:02:04.309445 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0c2bfb41-6e54-4117-9a7f-25d0cbc6e66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:04.318732 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1429/2884] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:02:04.323617 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678af92-0b20-4d56-b45e-07c3e8a9b596 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:04.333730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678af92-0b20-4d56-b45e-07c3e8a9b596 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:04.339912 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4511dd-76e7-4a85-a718-41149769186f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:04.340541 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4511dd-76e7-4a85-a718-41149769186f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.340819 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4511dd-76e7-4a85-a718-41149769186f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.341081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4511dd-76e7-4a85-a718-41149769186f admin admin] PATH_INFO: `/v3/domains` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.341771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4511dd-76e7-4a85-a718-41149769186f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80169) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:02:04.342107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4511dd-76e7-4a85-a718-41149769186f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:04.343705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4511dd-76e7-4a85-a718-41149769186f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:04.356430 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1456/2885] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:02:04.361443 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bfb41-6e54-4117-9a7f-25d0cbc6e66d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:04.376015 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2bfb41-6e54-4117-9a7f-25d0cbc6e66d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:04.383977 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edbdad-4e16-4323-871f-ce30edb1f7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:04.384595 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edbdad-4e16-4323-871f-ce30edb1f7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.384873 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edbdad-4e16-4323-871f-ce30edb1f7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.385149 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edbdad-4e16-4323-871f-ce30edb1f7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.385850 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edbdad-4e16-4323-871f-ce30edb1f7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:04.387797 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edbdad-4e16-4323-871f-ce30edb1f7e3 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:04.420549 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1430/2886] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:02:04 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:02:04.424674 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4511dd-76e7-4a85-a718-41149769186f admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:04.435290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4511dd-76e7-4a85-a718-41149769186f admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:04.494367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145c67e-b2ea-4e94-80d9-86e4dc45c534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:04.494988 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145c67e-b2ea-4e94-80d9-86e4dc45c534 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.495266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145c67e-b2ea-4e94-80d9-86e4dc45c534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.495540 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145c67e-b2ea-4e94-80d9-86e4dc45c534 admin admin] PATH_INFO: `/v3/users` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.496194 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145c67e-b2ea-4e94-80d9-86e4dc45c534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:04.498146 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145c67e-b2ea-4e94-80d9-86e4dc45c534 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:04.509143 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:02:04.524279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1457/2887] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:02:04 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:02:04.528494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edbdad-4e16-4323-871f-ce30edb1f7e3 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:04.539050 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edbdad-4e16-4323-871f-ce30edb1f7e3 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:04.544715 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1e2cd-e609-43e2-88a5-6a321d1e9008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:04.545331 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e2cd-e609-43e2-88a5-6a321d1e9008 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.545612 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e2cd-e609-43e2-88a5-6a321d1e9008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.545920 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e2cd-e609-43e2-88a5-6a321d1e9008 admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.546622 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1e2cd-e609-43e2-88a5-6a321d1e9008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:04.548215 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1e2cd-e609-43e2-88a5-6a321d1e9008 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:04.558739 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1431/2888] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:02:04.563037 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145c67e-b2ea-4e94-80d9-86e4dc45c534 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:04.573984 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145c67e-b2ea-4e94-80d9-86e4dc45c534 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:04.578967 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aefdf8-8639-4f4f-b48e-de10f722d71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:04.579573 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aefdf8-8639-4f4f-b48e-de10f722d71a admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.579852 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aefdf8-8639-4f4f-b48e-de10f722d71a admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.580123 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aefdf8-8639-4f4f-b48e-de10f722d71a admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.580816 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aefdf8-8639-4f4f-b48e-de10f722d71a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:04.582442 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aefdf8-8639-4f4f-b48e-de10f722d71a admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:04.593038 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1458/2889] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:02:04.597254 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1e2cd-e609-43e2-88a5-6a321d1e9008 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:04.607508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1e2cd-e609-43e2-88a5-6a321d1e9008 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:04.612800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d312ad9-568a-439c-ab22-1a1cd6549818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:04.613467 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312ad9-568a-439c-ab22-1a1cd6549818 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.613776 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312ad9-568a-439c-ab22-1a1cd6549818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.614047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d312ad9-568a-439c-ab22-1a1cd6549818 admin admin] PATH_INFO: `/v3/projects/0cb9eddabdda4f189e94530a629acaa5/users/5c85ec0c35a74a1d9c8a8276a65512fe/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.629642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d312ad9-568a-439c-ab22-1a1cd6549818 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cb9eddabdda4f189e94530a629acaa5, user_id=5c85ec0c35a74a1d9c8a8276a65512fe, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:04.632347 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d312ad9-568a-439c-ab22-1a1cd6549818 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:04.647989 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1432/2890] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:02:04 2022] PUT /identity/v3/projects/0cb9eddabdda4f189e94530a629acaa5/users/5c85ec0c35a74a1d9c8a8276a65512fe/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:02:04.659489 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf79433-e459-4174-9778-a4a27c4ea424 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.659772 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf79433-e459-4174-9778-a4a27c4ea424 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.660041 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf79433-e459-4174-9778-a4a27c4ea424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.688493 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-adf79433-e459-4174-9778-a4a27c4ea424 None None] MFA Rules not processed for user `5c85ec0c35a74a1d9c8a8276a65512fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:02:04.689763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-adf79433-e459-4174-9778-a4a27c4ea424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:04.762226 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1459/2891] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:02:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 26 08:02:04.774385 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d312ad9-568a-439c-ab22-1a1cd6549818 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:04.776256 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-6d312ad9-568a-439c-ab22-1a1cd6549818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:04.839469 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d312ad9-568a-439c-ab22-1a1cd6549818 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:04.843510 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749abc0-6d6b-42de-ab7e-a0781fa4a77d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c6943686c843e588df14f7c6dffb8a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:04.844167 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749abc0-6d6b-42de-ab7e-a0781fa4a77d service glance] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:04.844451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749abc0-6d6b-42de-ab7e-a0781fa4a77d service glance] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:04.844722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749abc0-6d6b-42de-ab7e-a0781fa4a77d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:04.853936 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749abc0-6d6b-42de-ab7e-a0781fa4a77d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:04.855376 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749abc0-6d6b-42de-ab7e-a0781fa4a77d service glance] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:04.864339 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1433/2892] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:02:04 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:02:14.758983 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=39 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:02:14.860320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:02:31.758042 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf79433-e459-4174-9778-a4a27c4ea424 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:31.823888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf79433-e459-4174-9778-a4a27c4ea424 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:31.829680 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:31.830404 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:31.830681 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:31.830950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] PATH_INFO: `/v3/users/5c85ec0c35a74a1d9c8a8276a65512fe` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:31.832321 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c85ec0c35a74a1d9c8a8276a65512fe)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:31.834298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:31.882328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c85ec0c35a74a1d9c8a8276a65512fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:31.882953 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c85ec0c35a74a1d9c8a8276a65512fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:31.896349 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c85ec0c35a74a1d9c8a8276a65512fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:31.905014 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c85ec0c35a74a1d9c8a8276a65512fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:31.913586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c85ec0c35a74a1d9c8a8276a65512fe'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:31.920831 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1460/2893] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:02:31 2022] DELETE /identity/v3/users/5c85ec0c35a74a1d9c8a8276a65512fe => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:02:31.925552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1b1b0-7596-4e57-a555-f08ca555839e None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:31.926083 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1b1b0-7596-4e57-a555-f08ca555839e None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:31.926419 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f1b1b0-7596-4e57-a555-f08ca555839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:31.948948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-51f1b1b0-7596-4e57-a555-f08ca555839e None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:02:31.950202 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51f1b1b0-7596-4e57-a555-f08ca555839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:32.014755 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1434/2894] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:02:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:02:32.025001 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:32.026650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:32.087668 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b2d87-bd71-408b-b542-da836d21ce1b admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:32.091679 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3fa07-1aeb-4c14-84f3-0eccba14a18e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:32.092353 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3fa07-1aeb-4c14-84f3-0eccba14a18e service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:32.092636 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3fa07-1aeb-4c14-84f3-0eccba14a18e service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:32.092900 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b3fa07-1aeb-4c14-84f3-0eccba14a18e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:32.102275 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3fa07-1aeb-4c14-84f3-0eccba14a18e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:32.103695 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b3fa07-1aeb-4c14-84f3-0eccba14a18e service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:32.113339 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1461/2895] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:02:32.397970 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1b1b0-7596-4e57-a555-f08ca555839e None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:32.399521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-51f1b1b0-7596-4e57-a555-f08ca555839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:32.412509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f1b1b0-7596-4e57-a555-f08ca555839e None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:32.418039 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:32.418699 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:32.418982 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:32.419255 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] PATH_INFO: `/v3/projects/0cb9eddabdda4f189e94530a629acaa5` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:32.420538 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cb9eddabdda4f189e94530a629acaa5)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:32.422509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:32.479207 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cb9eddabdda4f189e94530a629acaa5', 'request_id': 'req-d63a7961-6764-44ce-85ee-8c77553e2463'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:32.484869 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cb9eddabdda4f189e94530a629acaa5', 'request_id': 'req-d63a7961-6764-44ce-85ee-8c77553e2463'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:32.494539 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cb9eddabdda4f189e94530a629acaa5', 'request_id': 'req-d63a7961-6764-44ce-85ee-8c77553e2463'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:32.507065 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] The token cache is being invalidate because project 0cb9eddabdda4f189e94530a629acaa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80170) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:02:32.507349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:32.507991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-d63a7961-6764-44ce-85ee-8c77553e2463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:02:32.509535 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1435/2896] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:02:32 2022] DELETE /identity/v3/projects/0cb9eddabdda4f189e94530a629acaa5 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:02:42.109324 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:02:42.504143 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:02:54.896127 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3fa07-1aeb-4c14-84f3-0eccba14a18e service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:54.898137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-05b3fa07-1aeb-4c14-84f3-0eccba14a18e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:54.975686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b3fa07-1aeb-4c14-84f3-0eccba14a18e service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:55.037909 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708edf1-99a8-4af2-b266-f0c6701ae9b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:55.038537 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708edf1-99a8-4af2-b266-f0c6701ae9b8 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:55.038820 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708edf1-99a8-4af2-b266-f0c6701ae9b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:55.039093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5708edf1-99a8-4af2-b266-f0c6701ae9b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:55.040855 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5708edf1-99a8-4af2-b266-f0c6701ae9b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:55.052607 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708edf1-99a8-4af2-b266-f0c6701ae9b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:55.054168 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5708edf1-99a8-4af2-b266-f0c6701ae9b8 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:55.064827 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1462/2897] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:02:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:02:56.307714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baaf2a-ea8a-44a5-b20e-47bd82beb095 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:56.308121 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baaf2a-ea8a-44a5-b20e-47bd82beb095 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:56.308264 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4baaf2a-ea8a-44a5-b20e-47bd82beb095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:56.328707 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-d4baaf2a-ea8a-44a5-b20e-47bd82beb095 None None] MFA Rules not processed for user `77fb7ab7232b48a286b7798beca19f58`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:02:56.329937 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4baaf2a-ea8a-44a5-b20e-47bd82beb095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:56.385557 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1436/2898] 10.222.0.39 () {58 vars in 967 bytes} [Tue Jul 26 08:02:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:02:56.430816 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708edf1-99a8-4af2-b266-f0c6701ae9b8 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:02:56.432295 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-5708edf1-99a8-4af2-b266-f0c6701ae9b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:02:56.504548 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5708edf1-99a8-4af2-b266-f0c6701ae9b8 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:02:56.509905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d19d93-9dea-4bca-bcd4-1efdc0747cfd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '64257d34b2df4af1a2016d56be8d44a8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:02:56.510535 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d19d93-9dea-4bca-bcd4-1efdc0747cfd service swift] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:02:56.510812 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d19d93-9dea-4bca-bcd4-1efdc0747cfd service swift] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:02:56.511081 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d19d93-9dea-4bca-bcd4-1efdc0747cfd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:02:56.519638 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d19d93-9dea-4bca-bcd4-1efdc0747cfd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:02:56.521219 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d19d93-9dea-4bca-bcd4-1efdc0747cfd service swift] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:02:56.529896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1463/2899] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Jul 26 08:02:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 26 08:03:02.973074 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baaf2a-ea8a-44a5-b20e-47bd82beb095 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:02.974559 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d4baaf2a-ea8a-44a5-b20e-47bd82beb095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:03.039391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4baaf2a-ea8a-44a5-b20e-47bd82beb095 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:03.046111 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:03.046765 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:03.047047 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:03.047321 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] PATH_INFO: `/v3/users/dc6c885fee5440799beac8cfd3e42822` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:03.048507 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc6c885fee5440799beac8cfd3e42822)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:03.050486 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:03.098455 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc6c885fee5440799beac8cfd3e42822'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.099123 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc6c885fee5440799beac8cfd3e42822'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.111391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc6c885fee5440799beac8cfd3e42822'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.119060 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc6c885fee5440799beac8cfd3e42822'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.128722 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc6c885fee5440799beac8cfd3e42822'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.134961 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1437/2900] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:03:02 2022] DELETE /identity/v3/users/dc6c885fee5440799beac8cfd3e42822 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 26 08:03:03.315928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d19d93-9dea-4bca-bcd4-1efdc0747cfd service swift] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:03.327263 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d19d93-9dea-4bca-bcd4-1efdc0747cfd service swift] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:03.336938 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=100 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:03:03.387402 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:03.388059 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:03.388328 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:03.388600 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] PATH_INFO: `/v3/projects/8a46af50be774973800f747cdaf66896` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:03.389830 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a46af50be774973800f747cdaf66896)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:03.391748 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:03.440793 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a46af50be774973800f747cdaf66896', 'request_id': 'req-ebf91314-17fd-4e53-8583-70cb9068e4eb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.447650 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a46af50be774973800f747cdaf66896', 'request_id': 'req-ebf91314-17fd-4e53-8583-70cb9068e4eb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.453479 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a46af50be774973800f747cdaf66896', 'request_id': 'req-ebf91314-17fd-4e53-8583-70cb9068e4eb'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.467822 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] The token cache is being invalidate because project 8a46af50be774973800f747cdaf66896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:03:03.468107 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.468729 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.470304 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1464/2901] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:03:03 2022] DELETE /identity/v3/projects/8a46af50be774973800f747cdaf66896 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:03:03.474447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:03.475971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:03.489509 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e2877-9c33-4a74-86a2-d35ba9e3fadf admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:03.511391 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:03:03.559817 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:03.560601 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:03.560953 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:03.561330 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] PATH_INFO: `/v3/users/a31db786f09b4719a164010ff7086d11` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:03.562652 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a31db786f09b4719a164010ff7086d11)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:03.564552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:03.616300 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a31db786f09b4719a164010ff7086d11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.616941 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a31db786f09b4719a164010ff7086d11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.629426 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a31db786f09b4719a164010ff7086d11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.636951 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a31db786f09b4719a164010ff7086d11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.646463 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-cd63806d-3e7a-46dc-a5c8-94e5878475ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a31db786f09b4719a164010ff7086d11'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.652623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1438/2902] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:03:03 2022] DELETE /identity/v3/users/a31db786f09b4719a164010ff7086d11 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:03:03.838406 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:03.839965 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:03.855629 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf91314-17fd-4e53-8583-70cb9068e4eb admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:03.924185 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:03.924850 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:03.925135 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:03.925408 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] PATH_INFO: `/v3/projects/0c63ce137e4c4ba6a168dc2bddae5b25` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:03.926845 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c63ce137e4c4ba6a168dc2bddae5b25)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:03.928888 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:03.982546 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:03:03.986063 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c63ce137e4c4ba6a168dc2bddae5b25', 'request_id': 'req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:03.994481 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c63ce137e4c4ba6a168dc2bddae5b25', 'request_id': 'req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:04.000558 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c63ce137e4c4ba6a168dc2bddae5b25', 'request_id': 'req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:04.015203 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] The token cache is being invalidate because project 0c63ce137e4c4ba6a168dc2bddae5b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:03:04.015485 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:04.016177 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6c9118dc-bbf8-49c9-9beb-bf7f30de8829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:03:04.017921 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1465/2903] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:03:03 2022] DELETE /identity/v3/projects/0c63ce137e4c4ba6a168dc2bddae5b25 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:03:04.023061 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0bd5-5bc5-4f1e-a423-2ebf06f84053 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:04.023508 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0bd5-5bc5-4f1e-a423-2ebf06f84053 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:04.023620 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07d0bd5-5bc5-4f1e-a423-2ebf06f84053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:04.044682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-c07d0bd5-5bc5-4f1e-a423-2ebf06f84053 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:03:04.046199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c07d0bd5-5bc5-4f1e-a423-2ebf06f84053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:04.110451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1439/2904] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:03:04.118401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1be7f-652d-4f4a-8429-fb238212bf0c None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:04.118686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1be7f-652d-4f4a-8429-fb238212bf0c None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:04.118955 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b1be7f-652d-4f4a-8429-fb238212bf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:04.139825 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-08b1be7f-652d-4f4a-8429-fb238212bf0c None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:03:04.141030 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-08b1be7f-652d-4f4a-8429-fb238212bf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:04.150463 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1466/2905] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:03:04.154768 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d0bd5-5bc5-4f1e-a423-2ebf06f84053 None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:04.165075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07d0bd5-5bc5-4f1e-a423-2ebf06f84053 None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:04.171002 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a04a60-5c9c-4801-82eb-bcea00c0aa84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:04.171608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a04a60-5c9c-4801-82eb-bcea00c0aa84 admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:04.171882 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a04a60-5c9c-4801-82eb-bcea00c0aa84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:04.172151 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a04a60-5c9c-4801-82eb-bcea00c0aa84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:04.172852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a04a60-5c9c-4801-82eb-bcea00c0aa84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80170) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 26 08:03:04.173152 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a04a60-5c9c-4801-82eb-bcea00c0aa84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:04.174790 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a04a60-5c9c-4801-82eb-bcea00c0aa84 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:04.186948 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1440/2906] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Jul 26 08:03:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 26 08:03:04.191276 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1be7f-652d-4f4a-8429-fb238212bf0c None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:04.200875 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b1be7f-652d-4f4a-8429-fb238212bf0c None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:04.206060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0a758-37fb-491c-a2df-f078a18f8ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:04.206667 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0a758-37fb-491c-a2df-f078a18f8ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:04.206945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0a758-37fb-491c-a2df-f078a18f8ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:04.207183 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a0a758-37fb-491c-a2df-f078a18f8ba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:04.207873 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0a758-37fb-491c-a2df-f078a18f8ba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:04.209776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a0a758-37fb-491c-a2df-f078a18f8ba4 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:04.242533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1467/2907] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:03:04 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 26 08:03:04.248312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a04a60-5c9c-4801-82eb-bcea00c0aa84 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:04.258723 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a04a60-5c9c-4801-82eb-bcea00c0aa84 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:04.287457 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:03:04.325693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc7515-621b-4b5a-b795-10ff06f733af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:04.326335 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7515-621b-4b5a-b795-10ff06f733af admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:04.326612 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7515-621b-4b5a-b795-10ff06f733af admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:04.326938 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7515-621b-4b5a-b795-10ff06f733af admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:04.327607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc7515-621b-4b5a-b795-10ff06f733af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:04.329583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc7515-621b-4b5a-b795-10ff06f733af admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:04.352693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1441/2908] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:03:04 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:03:04.356933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0a758-37fb-491c-a2df-f078a18f8ba4 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:04.367730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a0a758-37fb-491c-a2df-f078a18f8ba4 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:04.372754 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6b36e-2dae-491d-b231-d7793f59e3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:04.373367 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6b36e-2dae-491d-b231-d7793f59e3d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:04.373634 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6b36e-2dae-491d-b231-d7793f59e3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:04.373960 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6b36e-2dae-491d-b231-d7793f59e3d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:04.374663 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6b36e-2dae-491d-b231-d7793f59e3d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:04.376345 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6b36e-2dae-491d-b231-d7793f59e3d5 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:04.386496 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1468/2909] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 26 08:03:04.391115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc7515-621b-4b5a-b795-10ff06f733af admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:04.407287 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc7515-621b-4b5a-b795-10ff06f733af admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:04.416075 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb850e39-0ecd-4704-b151-70c9c631d8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:04.416930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb850e39-0ecd-4704-b151-70c9c631d8eb admin admin] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:04.417222 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb850e39-0ecd-4704-b151-70c9c631d8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:04.417511 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb850e39-0ecd-4704-b151-70c9c631d8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:04.418600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb850e39-0ecd-4704-b151-70c9c631d8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:04.420892 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb850e39-0ecd-4704-b151-70c9c631d8eb admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:04.431332 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1442/2910] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:03:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:03:04.435591 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6b36e-2dae-491d-b231-d7793f59e3d5 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:04.447469 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6b36e-2dae-491d-b231-d7793f59e3d5 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:04.453342 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cb400-ea10-4596-bae6-47e66aa24a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:04.454454 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cb400-ea10-4596-bae6-47e66aa24a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:04.454741 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cb400-ea10-4596-bae6-47e66aa24a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:04.455099 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077cb400-ea10-4596-bae6-47e66aa24a13 admin admin] PATH_INFO: `/v3/projects/491378f17ebd4fddbf00a3b55a2d5b85/users/7881b5c6627d41eca68a27fc38559b2f/roles/64dfc66a8abe4468ac4f1ca071d28bd4` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:04.479614 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cb400-ea10-4596-bae6-47e66aa24a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=491378f17ebd4fddbf00a3b55a2d5b85, user_id=7881b5c6627d41eca68a27fc38559b2f, role_id=64dfc66a8abe4468ac4f1ca071d28bd4)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:04.483446 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077cb400-ea10-4596-bae6-47e66aa24a13 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:04.498757 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1469/2911] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:03:04 2022] PUT /identity/v3/projects/491378f17ebd4fddbf00a3b55a2d5b85/users/7881b5c6627d41eca68a27fc38559b2f/roles/64dfc66a8abe4468ac4f1ca071d28bd4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:03:04.504686 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe3268-2369-4d3a-a09c-6964b5b92253 None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:04.505023 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe3268-2369-4d3a-a09c-6964b5b92253 None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:04.505412 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe3268-2369-4d3a-a09c-6964b5b92253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:04.528474 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-98fe3268-2369-4d3a-a09c-6964b5b92253 None None] MFA Rules not processed for user `9e725435b0294296a28451de9a0d08c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:03:04.529695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-98fe3268-2369-4d3a-a09c-6964b5b92253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:04.593991 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1443/2912] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:03:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:03:04.604026 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cb400-ea10-4596-bae6-47e66aa24a13 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:04.606100 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-077cb400-ea10-4596-bae6-47e66aa24a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:04.665210 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077cb400-ea10-4596-bae6-47e66aa24a13 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:04.669406 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f0c54-6cf4-484a-93e7-cc3c46a4879d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:04.670118 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f0c54-6cf4-484a-93e7-cc3c46a4879d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:04.670403 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f0c54-6cf4-484a-93e7-cc3c46a4879d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:04.670705 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014f0c54-6cf4-484a-93e7-cc3c46a4879d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:04.680174 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f0c54-6cf4-484a-93e7-cc3c46a4879d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:04.681615 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014f0c54-6cf4-484a-93e7-cc3c46a4879d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:04.691662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1470/2913] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:03:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:03:08.912526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb8ef0-b364-4497-808e-22069c2f332d None None] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:08.913016 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb8ef0-b364-4497-808e-22069c2f332d None None] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:08.913108 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bb8ef0-b364-4497-808e-22069c2f332d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:08.941447 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.auth.core [None req-92bb8ef0-b364-4497-808e-22069c2f332d None None] MFA Rules not processed for user `7881b5c6627d41eca68a27fc38559b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80170) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:03:08.942931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-92bb8ef0-b364-4497-808e-22069c2f332d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:09.014307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1444/2914] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:03:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:03:09.018642 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f0c54-6cf4-484a-93e7-cc3c46a4879d service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:09.028933 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014f0c54-6cf4-484a-93e7-cc3c46a4879d service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:09.033533 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824e4a2-d81f-4079-b94a-6e325693e2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:09.034257 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824e4a2-d81f-4079-b94a-6e325693e2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:09.034524 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824e4a2-d81f-4079-b94a-6e325693e2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:09.034797 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f824e4a2-d81f-4079-b94a-6e325693e2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:09.035472 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824e4a2-d81f-4079-b94a-6e325693e2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:09.037413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f824e4a2-d81f-4079-b94a-6e325693e2b6 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:09.064896 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1471/2915] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Jul 26 08:03:09 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:03:09.069007 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb8ef0-b364-4497-808e-22069c2f332d None None] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:09.081005 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bb8ef0-b364-4497-808e-22069c2f332d None None] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:09.120208 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:03:09.140660 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b07b45-9d8d-43ec-8f6f-96e4216e600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:09.141293 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b07b45-9d8d-43ec-8f6f-96e4216e600c admin admin] REQUEST_METHOD: `POST` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:09.141575 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b07b45-9d8d-43ec-8f6f-96e4216e600c admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:09.141968 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b07b45-9d8d-43ec-8f6f-96e4216e600c admin admin] PATH_INFO: `/v3/users` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:09.142650 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b07b45-9d8d-43ec-8f6f-96e4216e600c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:09.144571 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b07b45-9d8d-43ec-8f6f-96e4216e600c admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:09.168548 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1445/2916] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Jul 26 08:03:09 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 26 08:03:09.172763 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824e4a2-d81f-4079-b94a-6e325693e2b6 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:09.183181 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f824e4a2-d81f-4079-b94a-6e325693e2b6 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:09.188279 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e233a71-b348-470d-9175-a6ef2f2a8699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:09.188878 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e233a71-b348-470d-9175-a6ef2f2a8699 admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:09.189161 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e233a71-b348-470d-9175-a6ef2f2a8699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:09.189453 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e233a71-b348-470d-9175-a6ef2f2a8699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:09.190179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e233a71-b348-470d-9175-a6ef2f2a8699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:09.191783 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e233a71-b348-470d-9175-a6ef2f2a8699 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:09.202791 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1472/2917] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:03:09.208642 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b07b45-9d8d-43ec-8f6f-96e4216e600c admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:09.219544 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b07b45-9d8d-43ec-8f6f-96e4216e600c admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:09.225199 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873839f0-3ef0-4d89-9b77-78d2ed689677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:09.225931 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873839f0-3ef0-4d89-9b77-78d2ed689677 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:09.226205 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873839f0-3ef0-4d89-9b77-78d2ed689677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:09.226488 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873839f0-3ef0-4d89-9b77-78d2ed689677 admin admin] PATH_INFO: `/v3/projects/f752f9a399844b17a7dda8b8761baca5/users/c9222d8c1814423eb74e550c4fbc2f61/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:09.243375 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873839f0-3ef0-4d89-9b77-78d2ed689677 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f752f9a399844b17a7dda8b8761baca5, user_id=c9222d8c1814423eb74e550c4fbc2f61, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:09.246073 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873839f0-3ef0-4d89-9b77-78d2ed689677 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:09.261578 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1446/2918] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:03:09 2022] PUT /identity/v3/projects/f752f9a399844b17a7dda8b8761baca5/users/c9222d8c1814423eb74e550c4fbc2f61/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:03:09.265833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e233a71-b348-470d-9175-a6ef2f2a8699 admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:09.276290 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e233a71-b348-470d-9175-a6ef2f2a8699 admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:09.315137 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:03:09.335413 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc90ad-b761-450a-a3c2-0939ef3e8e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:09.336031 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc90ad-b761-450a-a3c2-0939ef3e8e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:09.336312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc90ad-b761-450a-a3c2-0939ef3e8e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:09.336586 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc90ad-b761-450a-a3c2-0939ef3e8e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:09.337286 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc90ad-b761-450a-a3c2-0939ef3e8e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:09.338929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc90ad-b761-450a-a3c2-0939ef3e8e4f admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:09.348997 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1473/2919] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Jul 26 08:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 26 08:03:09.353231 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873839f0-3ef0-4d89-9b77-78d2ed689677 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:09.363667 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873839f0-3ef0-4d89-9b77-78d2ed689677 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:09.368451 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfae65-fb16-4f5f-91df-e3a0051e2260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:09.369115 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfae65-fb16-4f5f-91df-e3a0051e2260 admin admin] REQUEST_METHOD: `PUT` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:09.369398 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfae65-fb16-4f5f-91df-e3a0051e2260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:09.369671 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bfae65-fb16-4f5f-91df-e3a0051e2260 admin admin] PATH_INFO: `/v3/projects/f752f9a399844b17a7dda8b8761baca5/users/c9222d8c1814423eb74e550c4fbc2f61/roles/c0cd7b1474694ed7989af245c9f34118` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:09.371695 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfae65-fb16-4f5f-91df-e3a0051e2260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f752f9a399844b17a7dda8b8761baca5, user_id=c9222d8c1814423eb74e550c4fbc2f61, role_id=c0cd7b1474694ed7989af245c9f34118)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:09.374414 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bfae65-fb16-4f5f-91df-e3a0051e2260 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:09.384754 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1447/2920] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Jul 26 08:03:09 2022] PUT /identity/v3/projects/f752f9a399844b17a7dda8b8761baca5/users/c9222d8c1814423eb74e550c4fbc2f61/roles/c0cd7b1474694ed7989af245c9f34118 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:03:13.770945 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f643cc-4bfd-4ead-93c6-c8881f951f8f None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:13.771428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f643cc-4bfd-4ead-93c6-c8881f951f8f None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:13.771514 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f643cc-4bfd-4ead-93c6-c8881f951f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:13.816428 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-98f643cc-4bfd-4ead-93c6-c8881f951f8f None None] MFA Rules not processed for user `c9222d8c1814423eb74e550c4fbc2f61`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:03:13.817833 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-98f643cc-4bfd-4ead-93c6-c8881f951f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:13.918103 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1474/2921] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:03:13 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:03:13.934144 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfae65-fb16-4f5f-91df-e3a0051e2260 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:13.936079 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d1bfae65-fb16-4f5f-91df-e3a0051e2260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:14.033433 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bfae65-fb16-4f5f-91df-e3a0051e2260 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:14.040311 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf0442-b4b9-4b7a-9d64-59d9a67820b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:14.040963 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf0442-b4b9-4b7a-9d64-59d9a67820b5 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:14.041242 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf0442-b4b9-4b7a-9d64-59d9a67820b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:14.041515 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf0442-b4b9-4b7a-9d64-59d9a67820b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:14.107247 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf0442-b4b9-4b7a-9d64-59d9a67820b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:14.109343 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bf0442-b4b9-4b7a-9d64-59d9a67820b5 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:14.120677 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1448/2922] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:03:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 26 08:03:23.914929 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=45 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:03:24.116082 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:03:30.473490 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f920229-55ca-491a-b38b-0ee2cf101fb8 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:30.474060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f920229-55ca-491a-b38b-0ee2cf101fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:30.474060 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f920229-55ca-491a-b38b-0ee2cf101fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:30.495688 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-4f920229-55ca-491a-b38b-0ee2cf101fb8 None None] MFA Rules not processed for user `7881b5c6627d41eca68a27fc38559b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:03:30.496943 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4f920229-55ca-491a-b38b-0ee2cf101fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:30.507021 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1475/2923] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:03:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:03:30.835139 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf0442-b4b9-4b7a-9d64-59d9a67820b5 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:30.893569 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf0442-b4b9-4b7a-9d64-59d9a67820b5 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:30.898307 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8713d0-e5c1-4686-8d93-a36fb5ad711d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:30.898921 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8713d0-e5c1-4686-8d93-a36fb5ad711d service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:30.899210 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8713d0-e5c1-4686-8d93-a36fb5ad711d service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:30.899480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8713d0-e5c1-4686-8d93-a36fb5ad711d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:30.901572 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3b8713d0-e5c1-4686-8d93-a36fb5ad711d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:30.912551 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8713d0-e5c1-4686-8d93-a36fb5ad711d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:30.914017 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8713d0-e5c1-4686-8d93-a36fb5ad711d service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:30.922615 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1449/2924] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:03:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:03:31.388085 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f920229-55ca-491a-b38b-0ee2cf101fb8 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:31.389643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-4f920229-55ca-491a-b38b-0ee2cf101fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:31.465206 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f920229-55ca-491a-b38b-0ee2cf101fb8 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:31.470020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996c48c-f94c-4e7a-9a44-b84ed62a2ab5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:31.470644 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996c48c-f94c-4e7a-9a44-b84ed62a2ab5 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:31.470914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996c48c-f94c-4e7a-9a44-b84ed62a2ab5 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:31.471182 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7996c48c-f94c-4e7a-9a44-b84ed62a2ab5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:31.472993 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7996c48c-f94c-4e7a-9a44-b84ed62a2ab5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:31.485324 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996c48c-f94c-4e7a-9a44-b84ed62a2ab5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:31.486928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7996c48c-f94c-4e7a-9a44-b84ed62a2ab5 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:31.496662 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1476/2925] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:03:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:03:33.332930 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8713d0-e5c1-4686-8d93-a36fb5ad711d service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:33.342280 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8713d0-e5c1-4686-8d93-a36fb5ad711d service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:33.347056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec174172-314f-4317-bd8a-cc51344276de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:33.347702 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec174172-314f-4317-bd8a-cc51344276de service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:33.347978 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec174172-314f-4317-bd8a-cc51344276de service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:33.348320 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec174172-314f-4317-bd8a-cc51344276de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:33.350434 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec174172-314f-4317-bd8a-cc51344276de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:33.367661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec174172-314f-4317-bd8a-cc51344276de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:33.367661 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec174172-314f-4317-bd8a-cc51344276de service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:33.373647 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1450/2926] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:03:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:03:41.492917 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=35 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:03:43.370223 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=32 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:03:46.922389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996c48c-f94c-4e7a-9a44-b84ed62a2ab5 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:03:46.923914 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-7996c48c-f94c-4e7a-9a44-b84ed62a2ab5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:03:46.935419 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7996c48c-f94c-4e7a-9a44-b84ed62a2ab5 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:03:46.939138 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8426a4-672b-46b4-baec-a61abeddf37d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:03:46.939746 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8426a4-672b-46b4-baec-a61abeddf37d service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:03:46.940023 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8426a4-672b-46b4-baec-a61abeddf37d service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:03:46.940294 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8426a4-672b-46b4-baec-a61abeddf37d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:03:46.950266 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8426a4-672b-46b4-baec-a61abeddf37d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:03:46.951635 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8426a4-672b-46b4-baec-a61abeddf37d service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:03:46.961027 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1477/2927] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:03:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:03:56.956905 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:05:14.314534 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec174172-314f-4317-bd8a-cc51344276de service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:05:14.316063 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-ec174172-314f-4317-bd8a-cc51344276de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:05:14.356811 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec174172-314f-4317-bd8a-cc51344276de service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:05:14.366582 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7e8ed-f1a4-4204-8bbf-0cdb8fbb57e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:05:14.367406 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7e8ed-f1a4-4204-8bbf-0cdb8fbb57e1 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:05:14.367711 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7e8ed-f1a4-4204-8bbf-0cdb8fbb57e1 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:05:14.368027 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7e8ed-f1a4-4204-8bbf-0cdb8fbb57e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:05:14.371211 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-4ef7e8ed-f1a4-4204-8bbf-0cdb8fbb57e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:05:14.384693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7e8ed-f1a4-4204-8bbf-0cdb8fbb57e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:05:14.386308 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7e8ed-f1a4-4204-8bbf-0cdb8fbb57e1 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:05:14.396383 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1451/2928] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:05:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:05:24.392384 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=14 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:05:45.931572 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a89c81-836b-4613-be7e-bc01bbafd450 None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:05:45.932083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a89c81-836b-4613-be7e-bc01bbafd450 None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:05:45.932083 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a89c81-836b-4613-be7e-bc01bbafd450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:05:45.962957 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-19a89c81-836b-4613-be7e-bc01bbafd450 None None] MFA Rules not processed for user `7881b5c6627d41eca68a27fc38559b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:05:45.964148 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-19a89c81-836b-4613-be7e-bc01bbafd450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:05:45.972178 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1478/2929] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:05:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:05:46.300036 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7e8ed-f1a4-4204-8bbf-0cdb8fbb57e1 service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:05:46.310268 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7e8ed-f1a4-4204-8bbf-0cdb8fbb57e1 service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:05:46.314630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb285d4-1561-4a88-b0d3-4c02ae31e642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:05:46.315241 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb285d4-1561-4a88-b0d3-4c02ae31e642 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:05:46.315526 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb285d4-1561-4a88-b0d3-4c02ae31e642 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:05:46.315800 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb285d4-1561-4a88-b0d3-4c02ae31e642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:05:46.317876 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-0eb285d4-1561-4a88-b0d3-4c02ae31e642 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:05:46.328183 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb285d4-1561-4a88-b0d3-4c02ae31e642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:05:46.329608 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb285d4-1561-4a88-b0d3-4c02ae31e642 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:05:46.337971 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1452/2930] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:05:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:05:55.958239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=7 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:05:56.334232 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:05:57.427771 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a89c81-836b-4613-be7e-bc01bbafd450 None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:05:57.452028 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a89c81-836b-4613-be7e-bc01bbafd450 None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:05:57.456750 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6a33b-3d00-419d-8e78-7601bc66faaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:05:57.457376 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6a33b-3d00-419d-8e78-7601bc66faaa service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:05:57.457652 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6a33b-3d00-419d-8e78-7601bc66faaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:05:57.457950 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6a33b-3d00-419d-8e78-7601bc66faaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:05:57.460061 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-2fb6a33b-3d00-419d-8e78-7601bc66faaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:05:57.501020 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6a33b-3d00-419d-8e78-7601bc66faaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:05:57.502501 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6a33b-3d00-419d-8e78-7601bc66faaa service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:05:57.522969 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1479/2931] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:05:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:05:58.703480 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb285d4-1561-4a88-b0d3-4c02ae31e642 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:05:58.723128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb285d4-1561-4a88-b0d3-4c02ae31e642 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:05:58.727824 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1ece9-7e61-4746-ad9e-66461ee2c043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:05:58.728442 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1ece9-7e61-4746-ad9e-66461ee2c043 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:05:58.728721 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1ece9-7e61-4746-ad9e-66461ee2c043 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:05:58.728996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1ece9-7e61-4746-ad9e-66461ee2c043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:05:58.730983 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-faa1ece9-7e61-4746-ad9e-66461ee2c043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:05:58.746496 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1ece9-7e61-4746-ad9e-66461ee2c043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:05:58.747916 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1ece9-7e61-4746-ad9e-66461ee2c043 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:05:58.759071 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1453/2932] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:05:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:06:07.521160 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=27 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:06:08.754472 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=11 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:07:40.499244 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9ce82-1cfb-4e2d-a014-9956e873d0ec None None] REQUEST_METHOD: `POST` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:07:40.499813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9ce82-1cfb-4e2d-a014-9956e873d0ec None None] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:07:40.499813 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e9ce82-1cfb-4e2d-a014-9956e873d0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:07:40.521443 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.auth.core [None req-f6e9ce82-1cfb-4e2d-a014-9956e873d0ec None None] MFA Rules not processed for user `7881b5c6627d41eca68a27fc38559b2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80169) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 26 08:07:40.522677 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-f6e9ce82-1cfb-4e2d-a014-9956e873d0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:07:40.531393 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1480/2933] 10.222.0.39 () {58 vars in 958 bytes} [Tue Jul 26 08:07:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 26 08:07:50.515755 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=3 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:07:54.009046 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1ece9-7e61-4746-ad9e-66461ee2c043 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:07:54.024345 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1ece9-7e61-4746-ad9e-66461ee2c043 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:07:54.031690 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9827fa5-e081-449d-850a-6ccd0efa51be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:07:54.032409 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9827fa5-e081-449d-850a-6ccd0efa51be service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:07:54.032682 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9827fa5-e081-449d-850a-6ccd0efa51be service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:07:54.033003 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9827fa5-e081-449d-850a-6ccd0efa51be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:07:54.035684 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-d9827fa5-e081-449d-850a-6ccd0efa51be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:07:54.065583 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9827fa5-e081-449d-850a-6ccd0efa51be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:07:54.067213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9827fa5-e081-449d-850a-6ccd0efa51be service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:07:54.078954 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1454/2934] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:07:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:07:57.838359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9ce82-1cfb-4e2d-a014-9956e873d0ec None None] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:07:57.854716 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e9ce82-1cfb-4e2d-a014-9956e873d0ec None None] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:07:57.862482 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51dc2b-27f7-4ddc-a7ab-d9cd7f7eb650 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:07:57.863151 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51dc2b-27f7-4ddc-a7ab-d9cd7f7eb650 service nova] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:07:57.863411 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51dc2b-27f7-4ddc-a7ab-d9cd7f7eb650 service nova] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:07:57.863678 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d51dc2b-27f7-4ddc-a7ab-d9cd7f7eb650 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:07:57.865473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-1d51dc2b-27f7-4ddc-a7ab-d9cd7f7eb650 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:07:57.878928 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51dc2b-27f7-4ddc-a7ab-d9cd7f7eb650 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:07:57.880389 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d51dc2b-27f7-4ddc-a7ab-d9cd7f7eb650 service nova] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:07:57.892204 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1481/2935] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:07:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:08:04.074401 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=11 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:08:07.887862 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:09:15.066213 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9827fa5-e081-449d-850a-6ccd0efa51be service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:09:15.088946 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9827fa5-e081-449d-850a-6ccd0efa51be service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:09:15.094312 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf66fd-9900-46e1-8399-6cc07351ed48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:09:15.094940 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf66fd-9900-46e1-8399-6cc07351ed48 service neutron] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:09:15.095221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf66fd-9900-46e1-8399-6cc07351ed48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:09:15.095491 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf66fd-9900-46e1-8399-6cc07351ed48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:09:15.109600 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf66fd-9900-46e1-8399-6cc07351ed48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:09:15.112864 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caf66fd-9900-46e1-8399-6cc07351ed48 service neutron] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:09:15.124802 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1455/2936] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:09:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:09:25.120996 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=10 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:09:25.156386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51dc2b-27f7-4ddc-a7ab-d9cd7f7eb650 service nova] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:09:25.168394 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d51dc2b-27f7-4ddc-a7ab-d9cd7f7eb650 service nova] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:09:25.173086 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b40d6-ce00-4165-a79b-4c23f9af1350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:09:25.173698 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b40d6-ce00-4165-a79b-4c23f9af1350 service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:09:25.174000 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b40d6-ce00-4165-a79b-4c23f9af1350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:09:25.174269 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b40d6-ce00-4165-a79b-4c23f9af1350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:09:25.183306 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b40d6-ce00-4165-a79b-4c23f9af1350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:09:25.184726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b40d6-ce00-4165-a79b-4c23f9af1350 service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:09:25.194202 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1482/2937] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:09:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:09:28.198611 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf66fd-9900-46e1-8399-6cc07351ed48 service neutron] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:09:28.212565 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf66fd-9900-46e1-8399-6cc07351ed48 service neutron] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:09:28.218973 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875a3dc-ce65-4811-9db2-a2a4cbf19327 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d50427014064f099af87932c77a849a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:09:28.219630 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875a3dc-ce65-4811-9db2-a2a4cbf19327 service nova] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:09:28.219927 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875a3dc-ce65-4811-9db2-a2a4cbf19327 service nova] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:09:28.220197 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875a3dc-ce65-4811-9db2-a2a4cbf19327 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:09:28.222128 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-9875a3dc-ce65-4811-9db2-a2a4cbf19327 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:09:28.235662 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875a3dc-ce65-4811-9db2-a2a4cbf19327 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:09:28.237221 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875a3dc-ce65-4811-9db2-a2a4cbf19327 service nova] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:09:28.247904 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1456/2938] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:09:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:09:35.189643 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=6 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:09:38.243771 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=6 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:09:38.680628 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b40d6-ce00-4165-a79b-4c23f9af1350 service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:09:38.689298 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b40d6-ce00-4165-a79b-4c23f9af1350 service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:09:38.693179 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aca788-b2a8-4ac4-a42f-3144688ed44b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1911678a39f749939dc8476617a884fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:09:38.693814 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aca788-b2a8-4ac4-a42f-3144688ed44b service neutron] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:09:38.694093 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aca788-b2a8-4ac4-a42f-3144688ed44b service neutron] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:09:38.694359 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aca788-b2a8-4ac4-a42f-3144688ed44b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:09:38.766282 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aca788-b2a8-4ac4-a42f-3144688ed44b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:09:38.767717 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aca788-b2a8-4ac4-a42f-3144688ed44b service neutron] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:09:38.787386 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1483/2939] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:09:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:09:41.914440 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875a3dc-ce65-4811-9db2-a2a4cbf19327 service nova] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:09:41.931494 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875a3dc-ce65-4811-9db2-a2a4cbf19327 service nova] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:09:41.938975 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e680d-5dc8-407c-9d31-b20d0ff6594d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:09:41.941371 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e680d-5dc8-407c-9d31-b20d0ff6594d service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:09:41.941717 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e680d-5dc8-407c-9d31-b20d0ff6594d service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:09:41.942493 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e680d-5dc8-407c-9d31-b20d0ff6594d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:09:41.950893 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-5d3e680d-5dc8-407c-9d31-b20d0ff6594d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:09:41.976902 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e680d-5dc8-407c-9d31-b20d0ff6594d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:09:41.979063 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e680d-5dc8-407c-9d31-b20d0ff6594d service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:09:41.992471 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1457/2940] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:09:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:09:44.863390 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aca788-b2a8-4ac4-a42f-3144688ed44b service neutron] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:09:44.875312 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aca788-b2a8-4ac4-a42f-3144688ed44b service neutron] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:09:44.880671 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e6686-5029-4f02-9c39-010f1816ecf0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:09:44.881366 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e6686-5029-4f02-9c39-010f1816ecf0 service placement] REQUEST_METHOD: `GET` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:09:44.881726 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e6686-5029-4f02-9c39-010f1816ecf0 service placement] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:09:44.882113 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e6686-5029-4f02-9c39-010f1816ecf0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:09:44.883994 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-359e6686-5029-4f02-9c39-010f1816ecf0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:09:44.896376 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e6686-5029-4f02-9c39-010f1816ecf0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:09:44.898065 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e6686-5029-4f02-9c39-010f1816ecf0 service placement] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:09:44.912131 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1484/2941] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:09:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:09:44.928552 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e680d-5dc8-407c-9d31-b20d0ff6594d service placement] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:09:44.953089 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e680d-5dc8-407c-9d31-b20d0ff6594d service placement] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:09:44.960030 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:09:44.960700 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:09:44.960987 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:09:44.961407 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] PATH_INFO: `/v3/users/7881b5c6627d41eca68a27fc38559b2f` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:09:44.962714 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7881b5c6627d41eca68a27fc38559b2f)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:09:44.964623 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:09:45.016297 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7881b5c6627d41eca68a27fc38559b2f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.016990 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7881b5c6627d41eca68a27fc38559b2f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.029939 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7881b5c6627d41eca68a27fc38559b2f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.037693 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7881b5c6627d41eca68a27fc38559b2f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.046386 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7881b5c6627d41eca68a27fc38559b2f'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.052607 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1458/2942] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:09:44 2022] DELETE /identity/v3/users/7881b5c6627d41eca68a27fc38559b2f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:09:45.231139 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e6686-5029-4f02-9c39-010f1816ecf0 service placement] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:09:45.244192 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e6686-5029-4f02-9c39-010f1816ecf0 service placement] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:09:45.302776 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:09:45.303355 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:09:45.303645 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:09:45.303932 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] PATH_INFO: `/v3/projects/491378f17ebd4fddbf00a3b55a2d5b85` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:09:45.305097 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=491378f17ebd4fddbf00a3b55a2d5b85)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:09:45.307040 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:09:45.354401 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '491378f17ebd4fddbf00a3b55a2d5b85', 'request_id': 'req-89915f44-80f8-4116-bd6e-184bb305b5bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.360357 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '491378f17ebd4fddbf00a3b55a2d5b85', 'request_id': 'req-89915f44-80f8-4116-bd6e-184bb305b5bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.365996 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '491378f17ebd4fddbf00a3b55a2d5b85', 'request_id': 'req-89915f44-80f8-4116-bd6e-184bb305b5bd'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.368398 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:09:45.392775 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] The token cache is being invalidate because project 491378f17ebd4fddbf00a3b55a2d5b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:09:45.393058 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.393672 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.395239 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1485/2943] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:09:45 2022] DELETE /identity/v3/projects/491378f17ebd4fddbf00a3b55a2d5b85 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:09:45.399349 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:09:45.400745 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:09:45.413629 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efbc06-cd3c-409d-88a5-e1d6ffbadba7 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:09:45.471229 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:09:45.471879 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:09:45.472152 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:09:45.472425 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] PATH_INFO: `/v3/users/c9222d8c1814423eb74e550c4fbc2f61` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:09:45.473648 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9222d8c1814423eb74e550c4fbc2f61)` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:09:45.475594 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:09:45.525113 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9222d8c1814423eb74e550c4fbc2f61'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.525764 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9222d8c1814423eb74e550c4fbc2f61'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.527664 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:09:45.540517 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9222d8c1814423eb74e550c4fbc2f61'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.548564 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9222d8c1814423eb74e550c4fbc2f61'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.556676 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.notifications [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9222d8c1814423eb74e550c4fbc2f61'} {{(pid=80170) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.562691 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1459/2944] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Jul 26 08:09:45 2022] DELETE /identity/v3/users/c9222d8c1814423eb74e550c4fbc2f61 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:09:45.756247 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] Authenticating user token {{(pid=80169) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:09:45.757867 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.fernet_utils [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80169) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:09:45.775744 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89915f44-80f8-4116-bd6e-184bb305b5bd admin admin] Validating token access rules against request {{(pid=80169) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:09:45.833216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9e725435b0294296a28451de9a0d08c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9aaf3bcccded4ffcb2a432d5c86ce8ea', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80169) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:09:45.833904 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:09:45.834216 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:09:45.834473 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] PATH_INFO: `/v3/projects/f752f9a399844b17a7dda8b8761baca5` {{(pid=80169) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:09:45.835686 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f752f9a399844b17a7dda8b8761baca5)` {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:09:45.837602 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] RBAC: Authorization granted {{(pid=80169) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:09:45.885675 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f752f9a399844b17a7dda8b8761baca5', 'request_id': 'req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.891777 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f752f9a399844b17a7dda8b8761baca5', 'request_id': 'req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.897354 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f752f9a399844b17a7dda8b8761baca5', 'request_id': 'req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55'} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.910388 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] The token cache is being invalidate because project f752f9a399844b17a7dda8b8761baca5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80169) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 26 08:09:45.910730 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.911310 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG keystone.notifications [None req-6d06bbe6-d1e2-4b73-9ab4-88b52c2fcf55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80169) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 26 08:09:45.912987 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: [pid: 80169|app: 0|req: 1486/2945] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Jul 26 08:09:45 2022] DELETE /identity/v3/projects/f752f9a399844b17a7dda8b8761baca5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 26 08:09:55.559337 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=11,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:09:55.907433 dv-cinder-850836-1-iscsi devstack@keystone.service[80169]: DEBUG dbcounter [-] [80169] Writing DB stats keystone:DELETE=8,keystone:SELECT=58,keystone:INSERT=1 {{(pid=80169) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 26 08:10:23.334056 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] Authenticating user token {{(pid=80170) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 26 08:10:23.335785 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:10:23.408521 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10779e-7e31-428f-a1d4-68ea67149445 admin admin] Validating token access rules against request {{(pid=80170) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 26 08:10:23.475226 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9f5b2-663f-4f2d-87ed-60c107244d49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51a5093a60ef45919ab6e159adf58336', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e283157691d420caa8d7a3a389eadf7', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80170) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 26 08:10:23.475852 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9f5b2-663f-4f2d-87ed-60c107244d49 service placement] REQUEST_METHOD: `GET` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 26 08:10:23.476140 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9f5b2-663f-4f2d-87ed-60c107244d49 service placement] SCRIPT_NAME: `/identity` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 26 08:10:23.476439 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9f5b2-663f-4f2d-87ed-60c107244d49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80170) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 26 08:10:23.478323 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.fernet_utils [None req-31a9f5b2-663f-4f2d-87ed-60c107244d49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80170) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 26 08:10:23.489586 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9f5b2-663f-4f2d-87ed-60c107244d49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 26 08:10:23.491194 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9f5b2-663f-4f2d-87ed-60c107244d49 service placement] RBAC: Authorization granted {{(pid=80170) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 26 08:10:23.501272 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: [pid: 80170|app: 0|req: 1460/2946] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Jul 26 08:10:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 26 08:10:33.497008 dv-cinder-850836-1-iscsi devstack@keystone.service[80170]: DEBUG dbcounter [-] [80170] Writing DB stats keystone:SELECT=58 {{(pid=80170) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}